MeetKai launches AI-powered metaverse, starting with a billboard in Times Square

MeetKai is a conversational AI company that has expanded into building a real-world metaverse, starting with a billboard in Times Square.

MeetKai is accessible via virtual reality as well as any smart device, and its version of the metaverse is akin to Pokémon Go creator Niantic’s view, as it blends the real world with the virtual.

MeetKai is the brainchild of CEO James Kaplan and billionaire Weili Dai, executive chair. They spent their early years at MeetKai building foundational AI technology, and now they’re marrying it with the metaverse.

They traveled to New York this week to unveil the MeetKai metaverse, which you can access on your phone via a QR code on a billboard in Times Square. When you tap on it with your smartphone camera app, the browser app generates an animated version of Times Square where you can roam about with your avatar. You can control where your avatar goes on the phone.

“The metaverse as we envision it is essentially mapping the real-world physical world to the AI virtual world,” Dai said in an interview with GamesBeat. “We apply AI technology and all of a sudden your experience and knowledge, anything you do, is going to be at the speed up the light. Once you have this super-fast, efficient experience where you apply what you learn to the physical world, you contribute to a better life.”

You use your smartphone to “move” your character around the square. There’s not much you can do yet, but it’s a start. But the company believes there isn’t a limit to the number of people it can render together in a place like Times Square, which sounds like an interesting trick.

Kaplan said that the company will white label the service to brands and other companies that want to use the metaverse technology to reach more people.

The company has raised more than $20 million to date and it has 40 employees. The demo of the tech looks very early . It might seem like a puny effort, as there are more than 500 companies going after the metaverse, which McKinsey believes could be worth $5 trillion by 2030.

But you have to pay attention to a metaverse company started by a billionaire.

“We’re very committed,” Dai said. “It’s a private company that will lead the world in innovation and we will scale it as fast as we can.”

In Times Square

Free and open to the public, a physical billboard at the corner of 7th Avenue and 47th Street has been turned into an anamorphic portal to MeetKai’s metaverse in a virtual recreation of Times Square in New York City.

After scanning a QR code, users can simultaneously explore the exact same location physically and
digitally, interact with other players, and enter other worlds from MeetKai’s metaverse.

This “Phase 1 Beta” activation is the first installment of MeetKai’s broader ambition to map the entire
world and enhance reality instead of trying to replace it, making activities such as shopping,
collaborating or learning easier and more enjoyable with the help of AI and VR, said Dai.

“We are rewriting the rules of the metaverse in ways that lower barriers to technology adoption, make
it easier and more accessible on-the-go for smart device users, and provide high-performance immersive
experiences; all without the need for closed system apps or costly hardware such as VR headsets,” said
Kaplan. “Our core expertise in conversational AI has also enabled us to develop leapfrog smart virtual assistant services that will make the metaverse more intuitive, human, and more valuable for consumers, creators, and businesses.” 

Using NFTs

While MeetKai’s metaverse access is open and free to anyone with the QR code, exclusive perks will be
provided for those who enter through the virtual portal located on a billboard at the corner of 7th
Avenue and 47th Street in Manhattan.

They will have the opportunity to claim rare “Key to the City – NYC Edition” non-fungible tokens (NFTs), which can authenticate unique digital items. With NFTs, MeetKai will let people unlock gift cards and special features. One of the keys will also grant its owner the chance to have a metaverse street named after them and inspired by their interests. The company plans to do a scavenger hunt with NFTs in New York as well.

MeetKai is using NFTs, which aren’t popular in some gaming circles. But NFTs are playing a supporting role and won’t necessarily be front and center, Kaplan said.

“We’re looking at applications of NFTs that make sense,” he said.

Access to MeetKai will be granted to everyone who scans the QR code today, and all users regardless of location will be able to claim a MeetKai Metaverse citizenship and enter an alternate Louvre metaverse, also developed by MeetKai. The Time Square portal and billboard will remain open until August 11, 2022.

Proven technology

MeetKai already has customers for its white-label AI services that reach more than 20 million users.

The company leverages an advanced patented conversational AI technology that is capable of understanding negation queries, natural speech and remembering context for authentic, multi-turn conversations.

MeetKai’s AI supports 16 languages and its personalized software solutions and virtual assistants are used by users across 30 countries in Europe, Asia, and South America.

“The foundation of the last few years of technology and all of these building blocks now can support the metaverse,” Kaplan said. “The goal is really to just create an entire world that matches our own, and then allow us to experiment there and customize things. The core concept is really the real world and to have very low friction interacting with it.”

Kaplan said the industry is in a dangerous state now as it is at the peak of its hype cycle.

“Unless people start delivering things that people can actually try and experience and see, even if it has rough edges, then the entire industry could fade away,” he said.

He pointed to the cryptocurrency meltdown as a real problem. But he said the fortunate thing is that the metaverse doesn’t need cryptocurrency to function.

The launch of MeetKai’s first voice-operated AI concierge enabled consumers to leave behind traditional command-based exchanges with smart device assistants, opting for purposeful and full-fledged interactions that feel natural enough to resemble being on a phone call with a real-life friend.

Dai refers to Kaplan as a software genius Kaplan has had a passion for AI for a long time and he started coding when he was six and wrote his first bots when he was nine. Kaplan serves as CEO, and he is an AI researcher, engineer, and inventor.

“He’s an innovator and inventor as well. So I’m very proud of him,” Dai said.

Dai is formidable as well. She became wealthy as the cofounder of Marvell Technology Group, which became a major semiconductor company. She was named by Newsweek as one of the 150 Women Who Shake the World, but she also had some hard times. She and her husband Sehat Sutardja were fired from Marvell over an accounting scandal at Marvell six years ago. No outright fraud was uncovered, but the regulators said there was constant sales pressure to hit targets.

Dai has since bounced back as a prolific investor.

An open and include metaverse, rooted in reality

Experiencing the metaverse today requires app downloads that act as closed systems and resources
such as VR headsets that aren’t accessible to all, the company said.

This reality has created a fragmentation and digital divide that could prevent entire populations from accessing and participating in the metaverse at its most formative stage of development. 

MeetKai Metaverse creates VR-based experiences designed for headsets. But it emphasizes open, intuitive tech that can be accessible to more than 83% of the world’s population owning a smartphone, as well as any browser-compatible device — tablets or PCs — with zero download time.

“What we’re trying to demonstrate, and what we’re really leaning into heavily, is the browser. It’s been forgotten about by a lot of people. But to me, it’s the coolest thing that you don’t download an app,” Kaplan said. “By being in the browser, it’s really low friction.”

“The AI is coming in handy creating the world in 3D,” Kaplan said. “The first piece of tech is how do you create scaled physical places you can interact with?”

As an example, you can’t quite interact with things like Apple Maps or Google Streetview. The AI also helps with asset optimization of the art.

“In game development, one of the hardest things is how you go from a high-quality assets that looks good to something that can actually exist and work with the user.”

The company is trying optimize images to work with the web browser, and that’s a lot harder. The company uses WebGL 2.0 to depict 3D in a web browser. Kaplan said the company hopes to open source its next-generation renderer. The browser strategy makes the company hardware agnostic, Dai said.

“We are doing real-time, dynamic, global, and hardware-independent tech,” she said. “This is real-time 3D rendering.”

MeetKai’s AI also ensures that communication mimics the way people converse in real life, versus
having to adapt to the way machines process information, the company said.

“Our vision for the MeetKai Metaverse is to map the real world on an AI-powered virtual world with rich
experiences that have the potential to improve life and expand knowledge using cutting-edge technology. This is a tool that people can use to enrich their skill set, work, collaborate and engage with
brands and organizations. As a result, efficiency is multiplied and real world lifestyles become more
meaningful in a way that everyone can truly benefit from,” said Dai. “This is our mission for an inclusive and democratized metaverse as the world’s leading and most innovative metaverse company — one that is founded on proven AI technology leadership, global access and connections, and significant resources required for long-term execution and success.” 

There is still a lot of work to do.

“We’ve started building out a map of New York City. So right now, the Upper West Side is mostly mapped out. And the goal is to have the rest of the city fully mapped out so that people can explore the virtual world,” Kaplan said. “One of our biggest innovations is running an entire MMO in a browser. That’s kind of the first step. We are using the browser as the delivery platform.”

That means, of course, the graphics for the MeetKai metaverse are going to look OK on a smartphone and better in a VR headset, but it won’t come near the visual quality of high-end video games. The experience could be similar to Niantic’s Pokémon Go, which is also accessible to a lot of people but falls short as an awesome graphical experience.

Kaplan said his company wanted to go farther than just putting an augmented reality layer on top of a flat world, as Pokemon Go does. Rather, MeetKai takes a 3D map version of the world. (In a demo, I was running around in such a 3D version of Times Square).

Where they met the metaverse

Dai liked the concept of the metaverse because it matches with her philosophy of using technology for a better life.

“When we develop technology, it’s for a better life, to empower consumers around the world,” she said. “If you think about the metaverse, it’s such a rich technology with AI, global search, conversational multi-language AI, and the beauty is you can take people to other places in the real world, virtually.”

Kaplan started thinking more about the metaverse last year when he saw people proposing virtual worlds with no connection to the real world.

“The reason we started working on the MeetKai Metaverse last year was that we are a deep technology company,” Kaplan said. “We’re not really like a gaming company, and we haven’t really even done many consumer products. Most of our work has been pretty deep technology developments for search, virtualization, recommendation algorithms, voice assistants, and speech recognition.”

But last year, as the buzz around the metaverse grew, Kaplan and his team disagreed with the movement away from the real world to the virtual world. He felt that blending the real world and the metaverse was the right move.

“Everyone’s going to have their own world, it’s going to be just theirs,” he said. “Everyone’s going to have like their own little island that’s going to be theirs. I see that people are pushing for that angle. And I think that is really missing out on the real world. And that’s why we kind of went into this so heavily. We see this as this big hole where everyone has that mentality which is wrong.”

And the metaverse work will blend with the AI foundation soon.

“Some of the secret sauce we haven’t announced yet, but we’re going to tease that our most core technology is around conversational AI,” Kaplan said. “We believe our model is quite superior to GPT3 (an AI tech from Open AI Foundation) for powering non-player characters (NPCs). We’re going to have NPCs inside our world. There are other people mapping the world, but it’s going to be very hard to catch up with us on NPCs.”