Roblox shows its vision for UGC and generative AI

Roblox showed off its vision for the combination of user-generated content and generative AI.

Dan Sturman, CTO of Roblox, said in a blog post that the company sees an opportunity to use generative AI techniques to revolutionize creation on its platform.

Making it easier for creators to build games is part of its mission, and Roblox provides creators with a platform that enables end-to-end tools, services and support to help them build the most immersive 3D experiences, Sturman said. For certain, there is pressure for Roblox to respond to the wave of excitement around generative AI and show that it isn’t going to be passed by in this technology wave.

With Roblox Studio, creators have the chance to use out-of-the-box and free tools to build their experiences and publish immediately on all popular platforms, reaching as many as 58.8 million people daily worldwide, he said.

“With the advent of generative AI techniques, however, we are seeing an opportunity to revolutionize creation on the platform, both by augmenting Roblox Studio to make creation dramatically faster and easier, and also by enabling every user on Roblox to be a creator,” Sturman said.

Sturman said generative AI learns the underlying patterns and structures of data and generates new content, such as images, audio, code, text, 3D models or other forms of media, that have not been seen before.

“With a dramatic acceleration in these tools’ effectiveness for everyday content creation, this technology is at an inflection point,” he said. “It now has the capability to capture the creator’s intent, provide a broad range of digital editing capabilities, help create the content and allow for fast iteration. We have already heard from Roblox creators about how they are using this technology to create.”

However, he said these off-the-shelf AI systems are not integrated with our platform and they often do not produce “Roblox ready” output that requires substantial follow on work from a creator.

“We see an incredible opportunity to build generative AI tools and APIs focused on Roblox,” he said.

The value of generative AI for creators

Today a Roblox experience is created through a combination of 3D objects of various forms (constructive solid geometry, avatars, meshes, terrain, etc.), connected in behavior through Lua scripting, and backed by a universal physics engine that provides core behavior on the platform, he said.

While the engine is an underlying portion of the platform, it is up to the creator to build or otherwise acquire everything in the experience, Sturman said.

“We see generative AI tools being applicable to each of these different creative processes. What we have already started to see through internal prototypes is that specialized generative AI tools can not only accelerate creator productivity, but can significantly drop the technical skill needed to bring ideas to life,” Sturman said.

For example, some creators know how to code, but may have limited experience creating high-fidelity 3D models. Others may be more experienced with model design, but less experienced with code. In both cases, we see a future in which even a beginner can get a running head start as they look to bring their imagination to life in a Roblox experience, he said.

Even more powerful, the convergence of media supported by generative AI will allow creators to develop integrated 3D objects that come with behavior built in. For example, a creator could design a car through a simple statement such as “A red, two seater, convertible sports car with front-wheel drive”. This new creation would both look like a red sports car but also have all the behavior coded into it to be driven through a 3D virtual world, he said.

The video shows how quickly you can create using just text prompts.

This work involves unique technical challenges as the company tackles the ability to generate 3D models with event handlers, an animation rig, and physical properties.

“This work is unprecedented because making interactive content requires a deeper understanding of the generated object,” he said. “With the breadth of immersive content opportunities on Roblox, we also have the unique opportunity to create a generative model for all types of content at once – image, code, 3D models, audio, avatar creation and more.”

Everyone becomes a creator

Roblox is standing by the notion that it is building a platform that will enable every user to be a creator – not just those comfortable with Roblox Studio and other 3D content creation tools.

“We believe that many experiences in Roblox will become creation experiences where one can create a new shirt, hat, an entire avatar; a house or even an entire experience – all from within another experience,” he said. “This vision requires a set of tools significantly more accessible to a typical user than exists in any environment today – things like voice and text or touch-based gestures rather than intricate mouse and keyboard movements.”

Generative AI tooling can help make creation intuitive and natural for users and be directly embedded into experiences, allowing any of the 58.8 million daily users to create unique content that can be shared across the platform, he said.

Roblox sees a huge opportunity for the AI community itself to become creators on the Roblox platform.

“By enabling 3rd party AI creation services to plug directly into Roblox (likely as a creation experience), we provide a mechanism for their unique intellectual property to be directly offered to Roblox users,” Sturman said.

For example, if you develop an AI model that builds the most expressive superhero characters based on a combination of text prompts, graphical queues and photo examples, that capability is something you should be able to offer directly to those Roblox users who want an incredible superhero avatar.

“We envision the community as a force multiplier for generative AI, creating an ecosystem that our creators and users can leverage to create content and tools more effectively,” he said.

This vision also involves some unique technical challenges, he said. First, the tools have to be suited to things a user would create, and how they might fit into an in-experience creation environment. Second, in all cases, the company needs to keep Roblox safe and civil.

This means it needs to build a fast and scalable moderation flow for all types of creation. Third, it needs an economic system that encourages such creation. Roblox stands apart as a platform with a robust creator-backed marketplace and economy, and it must extend that to support in-experience user-creators as well as AI algorithm developers, he said.

“As we do this, we remain aware of the need to implement generative AI thoughtfully and ethically on the Roblox platform, in line with the value we have always placed on respecting our community. We are committed to using diverse and robust data sets to limit biased content and encourage safe and high-quality content output,” he said.

He said that Roblox with share more at his talk on the Tech Stack for the Metaverse at Roblox’s upcoming DeveloperWeek. And Roblox’s Stef Corazza will also share more during his talk at the AI Summit of Game Developers Conference next month.