Are Roblox’s new AI coding and art tools the future of game development?

SAN FRANCISCO—At the Game Developers Conference Monday, Roblox rolled out a new set of AI tools designed to let the company's millions of player-creators create usable game code and in-game 2D surfaces using nothing but simple text descriptions. Head of Roblox Studio Stef Corazza told a packed audience at the conference that the release is a major step toward "democratizing" game creation, taking it from "the hands of the skilled few" and giving it to people "who were blocked by technical hurdles but had a great idea" that they were previously unable to express without highly specialized skills.

“Create a 3 by 3 grid of orbs”

The release of the Roblox Code Assist beta Monday morning certainly seems to have the potential to let users create simple code snippets with a minimum of effort. In an example Corazza presented at the conference, a user could ask the system to "make orb turn red and destroy after 0.3 seconds when player touches it." The system then generates a seven-line Lua function that does just that, based on a coder-defined orb object provided earlier in the code. Another prompt for a function to "create a 3 by 3 grid of orbs around orb" similarly generates a few lines of code to place a small grid of those objects in the game scene.

Corazza said that, just four months ago, it wasn't clear that this tool would work well enough for a public release today. But Roblox has taken advantage of advances in natural language code generation that have rolled out in just the last few weeks. The key to getting usable results for the company's Code Generator Beta, though, was fine-tuning that standard model with code from the Roblox platform itself. That crucial context "significantly increases the quality of output," he said. The need for context applies to coders using the tool, too, Corazza said. Asking the AI to generate code on an empty document is akin to asking a knowledge expert to take a test "in a completely white room where you didn't hear the question completely." In internal testing, though, Corazza said providing the AI tool with just three lines of sample code to start from increased the "acceptance rate" for the tool's suggestions by 50 percent over attempts that started with no such "context" code. For now, the main focus of the Code Generator Beta is to allow experienced coders to not "have to work on simple stuff," Corazza said, and to "help automate basic coding tasks so you can focus on creative work." In the future, though, Corazza said he sees a more chatbot-style interface that can be used as a learning tool, explaining how code works and documenting functions for those still learning the basics.

“Scene with a forest, a river, and a large rock”

Alongside the AI code generator, Roblox has also rolled out a Material Generator designed to automate the tedious process of creating the kind of flat art assets that are layered on top of the many 2D surfaces in a game world. This goes beyond the kind of basic image generation you'd get from a tool like Stable Diffusion. Roblox's tool also automatically layers a faux 3D normal map onto the surface, alongside other "maps," for attributes like albedo, roughness, and "metalness." Those attributes can then be used by the game engine for accurate lighting reflections and responses to other objects.

But this is just "step one" of the company's AI asset-generation plans, Corazza said. The next step is an AI system that can go beyond flat surfaces and create an entire "specific geometry" that can retexture a full 3D model or character completely. This is "a very hard problem to crack" because of the need to be aware of the full context of the object itself (e.g., where various body parts go on a living character), but Corazza said the team has seen "some early breakthroughs," and he's "confident it will land" eventually. The "holy grail" for this kind of tool, though, is something that can mimic a "specific game style" all at once, Corazza added. The idea would be to take just a few drawings from a concept artist and have the AI instantly generate an entire set of assets that are consistent with that style and with each other. In an "extreme example" of a future use case, Corazza suggested someone might be able to type "Scene with a forest, a river, and a large rock" and get a completely interactive, realistic 3D world matching the prompt. "It'll feel like nuclear fusion," he said. "I'll say two years [until it's ready]."

A long road ahead

While Roblox warns that its AI system still doesn't always "suggest perfect code," Corazza said an environment like Roblox is a natural place to play around with these kinds of early, imperfect generative test cases. Unlike self-driving cars—where any AI errors could have "massive consequences"—the "bar is a little bit lower" for AI-generated Roblox code and surfaces, he said. "No catastrophic events will happen if the generation isn't good—just click the button and create another one." Roblox is already planning ahead for the challenge of moderating what the company expects will be a massive deluge of AI-generated content in the future, though. The company will probably need to develop more automated tools to instantly moderate "things created at runtime by tens or hundreds of millions of players," Corazza said. Despite those challenges, Corazza was effusive about future waves of AI-powered game creation tools that will eventually "converge" around the generation of all of a game's assets—materials, code, 3D assets, terrain, audio, avatars, 3D scene, and images—from a single text prompt. Those future tools will be built more directly around capturing the "intent of the user," unlike today's focus on fine, granular control at the code/vertex level. If and when that happens, the highly technical job of "game developer" could look very different from how it does today.