NeurIPS 2022 honors MineDojo for playing Minecraft guided by a series of written commands. A paper describing MineDojo, Nvidia’s general AI agent that can perform actions from a series of written commands in Minecraft, won the “ Outstanding Datasets and Benchmarks ” scientific paper award at the 2022 NeurIPS ( Neural Information Processing Systems ) conference. See also: Nvidia: AI turns real-world videos into 3D renders
To train the MineDojo framework to play Minecraft, the researchers fed it 730,000 Minecraft videos from YouTube (with more than 2.2 billion words transcribed), 7,000 webpages from the Minecraft wiki, 340,000 Reddit posts , and 6.6 million Reddit comments describing how to play Minecraft. From this data, the Nvidia AI researchers created MineClip , a custom transformer model that associates various videos with specific actions within the Minecraft game. As a result, someone can tell a MineDojo agent what to do in the game using high-level language , such as “ find a pyramid in the desert ” or “ build a portal and enter it, ” and MineDojo will perform a series of steps necessary to make that happen in the game. See also: Video Game Accessibility Awards: Let’s See the Big Winners
MineDojo aims to create a flexible agent that can generalize learned actions and apply them differently in the game . As Nvidia writes: “While researchers have long trained autonomous AI agents in game environments such as StarCraft , Dota 2 , and CS:Go , these agents are typically specialized in only a few tasks . So our researchers turned to Minecraft , one of the most popular games in the world, to develop a scalable training framework for a general agent, one that can successfully perform a wide variety of open-ended tasks ( a task that has no right or wrong answer, but that allows the player to offer their own opinions and ideas or respond creatively ).” You can see examples of MineDojo in action on their website, as well as the MineDojo & MineClip codes available on Github . Source: arstechnica.com
