Generative AI is being used to generate immersive video-game worlds.Credit: Jens Schlueter/Getty
Generative artificial intelligence (AI) tools have upended creative industries from music and film to scientific publishing. Now they’re upending the world of video games, too.
In a study published in Nature on 19 February, a team of researchers reveals a generative AI engine that can create coherent, immersive video-game worlds that respond to player inputs1.
“I knew that this could be done, but it’s still impressive,” says Julian Togelius, a researcher at New York University specializing in AI in video games.
Last year, researchers at Google showed that an AI engine could simulate the 1990s video game DOOM. But the ability for generative AI tools to generate fresh ideas in complex worlds has been limited. When a team led by Microsoft game-intelligence researcher Katja Hoffman interviewed game developers to find out what they wanted in AI tools, they suggested engines that can generate gameplay sequences that remain consistent with the rules and physics of the game. They also wanted the ability to tweak outputs throughout development.
The team built and trained WHAM, a generative AI system focusing on these capabilities. The researchers took the online multiplayer game Bleeding Edge, developed by a Microsoft-owned studio named Ninja Theory, and extracted video frames and controller inputs from 500,000 anonymized game sessions, which equated to more than seven years of continuous play. They trained the engine on one-second slices of gameplay involving up to 1.6 billion parameters.
The team showed that WHAM could perform well in three crucial areas. It could act within the rules of the original game (consistency) and generate a range of gameplay slices (diversity), with anything that was added to the world by the user remaining there (persistency). Together, these features demonstrate WHAM’s potential for creative ideation, the authors say.
Plausible gameplay
Finally, they built a concept prototype, the WHAM Demonstrator, which provides an interface for users to prompt the AI engine. In one example, the researchers showed that they could feed a single promotional image for Bleeding Edge into the Demonstrator and it would generate plausible gameplay and controller inputs. They also added a function that allows developers to drag and drop imagery into the Demonstrator, which the AI engine then incorporates directly into the generated game world. Togelius says this is the most impressive thing to emerge from the paper.