Inworld AI joins metaverse innovation with AI-driven virtual characters

Inworld AI, a company that aims to develop a platform that enables users to create AI-driven virtual characters which can be used to populate virtual worlds, announced today that it has raised $7 million in seed funding.

In an exclusive interview, Inworld’s cofounder and CEO Ilya Gelfenbeyn explained that “Inworld AI is a platform for building, basically brains for virtual characters” to populate virtual environments, including the metaverse, VR, and AR worlds. “What we provide is a toolset that enables developers to add brains and build these characters for the world, for different types of environments.”

To successfully create immersive characters, Inworld AI attempts to mimic the cognitive abilities of the human by leveraging a mixture of AI technologies like natural language understanding and processing, optical character recognition, reinforcement learning, and conversational AI to develop sophisticated virtual characters — characters that can even respond to questions and carry on conversations.

Inworld AI isn’t developing a solution to design visual avatars, but instead aims to create an AI development platform that enables companies that produce digital avatars and virtual characters to add more advanced communication to their visual designs.

The end goal of the platform is to offer a platform that visual avatar providers and organizations can use to develop “characters that can interact naturally with wide-ranging and completely open dialog,” Gelfenbeyn said. Although, speech is just the tip of the iceberg in terms of the communicative capabilities of these AI characters.

As Gelfenbeyn notes, “Inworld characters should not be limited to speech only, but be able to interact with many of the modalities that humans use, such as facial gestures, body language, emotions, as well as physical interactions.”

Enhancing the metaverse experience with AI brains

“We structure our technology stack based on inspiration from the human brain. We have three main components: perception, cognition, and behavior. Perception is focused on input and understanding of the environment and other agents, using senses like audio and visual,”  Gelfenbeyn said.

To enable virtual characters to perceive the environment audibly and visually, the organization uses a complex mixture of speech-to-text, rules engines, natural language understanding, OCR, and event triggers.

The next component is cognition.“Cognition is about the internal states of the character, such as memory, emotion, personality, goals, and background,” he said. Here Inworld AI will use natural language processing, emotion recognition, reinforcement learning, and goal-directed conversational AI to enhance the cognitive abilities of virtual characters.

Finally, “behavior is about the output or interactions of the character, such as speech gestures, body language, and motion.” Technologies like state-of-the-art generative language models, reinforcement learning, and customized voice and emotion synthesis,” enable virtual characters to replicate human gestures and behaviors.

Together, these three components provide a solid framework for developers to build virtual characters that can respond in detail to natural language, perceive the digital environment, and offer significant interactions for users.

Investors include Kleiner Perkins, CRV, and Meta. Inworld AI’s launch is well-timed, with publicity for the metaverse at an all-time high following Facebook’s rebrand to Meta, and decision-makers eager to identify what solutions are available to interact with customers in the metaverse.

As Izhar Harmony, General Partner of CRV explained, “the team is growing rapidly, so now is an exciting time for people interested in VR, games, and virtual worlds to partner with and join the company, so they can be at the forefront of this rapidly growing space.”

New kid on the block 

Inworld AI is entering into the highly competitive space of AI and machine learning development and competing against established providers like Open AI, and Google AI, that let you create machine learning models, yet Inworld AI fulfills a unique gap in the market, as it provides a highly specialized solution for developing conversational AI for AI-driven virtual characters, rather than generic machine learning models.

At the same time, the AI solutions that Inworld AI is developing will enable virtual character creation that extends well beyond the complexity of AI-driven avatars like Pandora Bots and Soul Machines.

“Many existing companies have solutions that provide limited answers to script triggers and dialog. In fact, our team built one of the largest providers of such services (API.ai, acquired by Google and now known as Google Dialogflow) so we are very familiar with their capabilities,” Gelfenbeyn said.

“Other companies are beginning to experiment with new technologies (such as large language models) but we believe that these parts, while essential, only provide one piece of the stack necessary to really bring characters to life,” he said.

In other words, these solutions have only scratched the surface of human-AI interactions, and Inworld AI’s approach to replicate human cognition is designed to create much more intelligent virtual entities. While Inworld AI’s mission to build AI brains for virtual characters is ambitious, the team’s AI development pedigree speaks for itself.

Inworld AI’s founders include a swath of experts such as Gelfenbeyn who was previously the CEO of API.ai, chief technology officer Michael Ermolenko, who led machine learning development at API.ai and the Dialogflow NLU/AI team at Google, and product director Kylan Gibbs, who previously led product for applied generative language models at DeepMind.

With this experienced team, the organization is in a strong position to set the standard for interactive virtual characters. After all, “Widespread success of the metaverse and other immersive applications depends on how enveloping those experiences can be,” said Ilya Fushman, investment partner at Kleiner Perkins.

“Inworld AI is building the engine that enables businesses to provide that exciting depth of experience and captivate users. With the team’s track record in providing developers with the tools they need to build AI-fueled applications, we’re excited to support the company in building the future of immersive experiences,” Fushman explained.

Virtual characters are key for immersion

With the metaverse boom beginning to pick up steam, Inworld AI also has a unique role to play in providing providers with a toolset that they can use to create sophisticated virtual characters and create more compelling digital experiences for users. The level of immersion offered by these experiences will determine whether the metaverse lives or dies.

The types of experiences that developers can use Inworld AI to build are diverse. As Gelfenbeyn explained, “Immersive realities continue to accelerate, with an increasingly diverse and fascinating ecosystem of worlds and use cases.”

“Virtual spaces like Meta’s Horizon Worlds, Roblox, Fortnite, and others that offer unique experiences and enable users to exist in other worlds will also continue to see quick demand from businesses, offering everything from games to story content to new enterprise applications,” Gelfenbeyn said.

Although Gelfenbeyn noted that the technology is simply to enable providers to create a “native population” for the digital world to offer realistic experiences, the metaverse is also becoming a new channel that technical decision-makers can use to interact with customers in the future.

While complete, immersive realities with sophisticated virtual characters are a long way off, Inworld AI’s team’s knowledge of conversational AI will undoubtedly enable other providers to move closer toward building vibrant, virtually populated, and interactive digital worlds.