DeepMind reveals Genie 3 “world model” that creates real-time interactive simulations

While no one has figured out how to make money from generative artificial intelligence, that hasn’t stopped Google DeepMind from pushing the boundaries of what’s possible with a big pile of inference. The capabilities (and costs) of these models have been on an impressive upward trajectory, a trend exemplified by the reveal of Genie 3. A mere seven months after showing off the Genie 2 “foundational world model,” which was itself a significant improvement over its predecessor, Google now has Genie 3.

With Genie 3, all it takes is a prompt or image to create an interactive world. Since the environment is continuously generated, it can be changed on the fly. You can add or change objects, alter weather conditions, or insert new characters—DeepMind calls these “promptable events.” The ability to create alterable 3D environments could make games more dynamic for players and offer developers new ways to prove out concepts and level designs. However, many in the gaming industry have expressed doubt that such tools would help.

It’s tempting to think of Genie 3 simply as a way to create games, but DeepMind sees this as a research tool, too. Games play a significant role in the development of artificial intelligence because they provide challenging, interactive environments with measurable progress. That’s why DeepMind previously turned to games like Go and StarCraft to expand the bounds of AI.

World models take that to the next level, generating an interactive world frame by frame. This provides an opportunity to refine how AI models—including so-called “embodied agents”—behave when they encounter real-world situations. One of the primary limitations as companies work toward the goal of artificial general intelligence (AGI) is the scarcity of reliable training data. After piping basically every webpage and video on the planet into AI models, researchers are turning toward synthetic data for many applications. DeepMind believes world models could be a key part of this effort, as they can be used to train AI agents with essentially unlimited interactive worlds.

DeepMind says Genie 3 is an important advancement because it offers much higher visual fidelity than Genie 2, and it’s truly real-time. Using keyboard input, it’s possible to navigate the simulated world in 720p resolution at 24 frames per second. Perhaps even more importantly, Genie 3 can remember the world it creates.