Howie Mandel gets a digital twin from DeepBrain AI

Howie Mandel is stepping into the metaverse. DeepBrain AI has created a pretty realistic AI version of comedian and actor Mandel.

Deepbrain AI, based in South Korea and Palo Alto, California, calls its creation “AI Howie,” and it’s an interactive virtual human and digital twin for immersive and personalized fan experiences. AI Howie mentions VentureBeat and talks to me in the attached videos.

Unlike the “deepfakes” of Tom Cruise and other actors, the real Howie Mandel cooperated with DeepBrain AI to create the virtual human AI replica of the famous comedian, actor, host, and technology enthusiast. We used “virtual Paris” AI character at our recent MetaBeat event in San Francisco.

“I am equally thrilled, excited, and terrified to finally have the ability of showing up and doing things without going anywhere or doing anything,” said Mandel, in a statement. “Thank you, DeepBrain.”

DeepBrain AI applies deep learning technology to create hyper-realistic virtual humans through its AI Studios and the AI Human platforms. These virtual humans are digital twins of the real person, with the same appearance, voice, gestures, and subtle mannerisms. The AI Studios platform enables script-to-video software that synthesizes dynamic video content in seconds, producing the quickest and most
realistic AI-generated videos. The script-to-video editor makes it easy for customers to select a model and then make it say something based on a script. Within a minute or so the video is made.

This is a powerful communication and marketing tool for celebrities, professional athletes, news anchors, and even politicians. Before working with Howie Mandel, the DeepBrain AI team created digital twins of Premier League soccer superstar Son Heung-Min, multiple news anchors across Asia, and South Korean president Yoon Suk-yeol.

Joe Murphy, business development manager for DeepBrain AI, said in an interview with VentureBeat that the virtual Howie is a conversational model that you pepper with questions. DeepBrain AI designs and develops these virtual humans for the purpose of creating digital twins (like Howie Mandel), digital people, and avatars.

“We create models of real people,” Murphy said. “We also have completely synthetic virtual humans. That is what we’ll call digital people. And then avatars are just the basic Roblox type of avatars. But where our technology comes in with the digital twins is we go through a deep learning process to clone the person’s voice, their mannerisms, their face, the way their eyes move, the way their lips move.”

He added, “So we create what we call the digital twin of the real person with all the uniqueness of that person. Our mission is to use this technology that we’ve developed throughout Asia and bring it to America.”

In addition to the script-to-video capabilities, the company provides fully conversational experiences with its AI Human software. The AI Human solution enables fans to interact and engage with AI Howie by simply asking questions. For example, when asked, “What was your favorite act on AGT this season?” the AI Howie model responds in real-time to support interactive, fun, and engaging fan experiences.

AI Humans are available within mobile apps, web browsers, or voice-activated kiosks.

“Our vision is to humanize digital experiences and empower creative teams to generate immersive content at scale,” said Eric Jang, DeepBrain AI CEO. “Working with Howie Mandel was a fun experience, and we are excited to see how the AI Howie collaboration will connect with his fans worldwide.”

DeepBrain AI, (formerly Moneybrain), a conversational AI startup based in Seoul, South Korea, has raised $44 million in a series B round led by Korea Development Bank at a post-money valuation of $180 million. The company started in 2016 and it has raised $54 million to date. The company has 130 employees.

The AI is being used for AI news anchors in South Korea and China at four different television networks. The networks flag that the anchor is an AI avatar so that no one gets confused.

While multiple companies are working on virtual humans, DeepBrain AI’s avatars are hyperrealistic. One of Asia’s largest insurance companies is also using it, as is a “brand ambassador” for a soccer team.

“When we worked with Howie Mandel, we went down to his studio in Los Angeles, we provided a script, and fed our training data into our neural network,” Murphy said.

It took about a day to do a video shoot with Mandel and about three to four weeks of machine learning time on the computers to generate the first AI model.

Back in January, DeepBrain AI opened its office in Palo Alto, California, and it is talking to partners in Silicon Valley and the rest of the U.S. Over time, Murphy said that the hope is to create AI avatars in realistic 3D for the metaverse. In South Korea, kiosks are appearing in places like banks with both 2D avatars and 3D avatars.

Over time, Murphy said the avatars have gotten better at mannerisms, lip sync, and subtle gestures. The speed of real-time responses in conversations has also gotten faster. The company is talking about doing more with game companies and major brands.