Singulos Research unveils Perceptus AI for AR that understands the real world

Singulos is taking the wraps off Perceptus, a spatial technology that enables augmented reality experiences with better fidelity in the real world.

Vancouver, Canada-based Singulos believes it can enable a new class of compelling AR applications using its Perceptus Platform, which enables real-time comprehension and tracking of physical objects in arbitrary 3D environments.

I interviewed CEO Brad Quinton, who is an AI professor and researcher at the University of British Columbia, and he showed me a demo where we played a round of chess together.

“Perceptus is really all about giving augmented reality the understanding of objects in the environment,” Quinton said. “There are a lot of augmented reality applications, but they don’t do much of anything. You can put a vase on a table, or pretend that you have a piece of furniture in the corner. But the apps don’t react to the world around you. And that’s why we believe that giving AR the ability to understand these objects allows it to understand your context. And using that context, it can be proactive and help you interact with the world.”

To show what he meant, he used an iPad to capture a view of his chessboard in the real world. He moved a piece on his real chessboard, and then I looked at my screen and saw the piece move on my screen. I used my mouse to move a pawn, and he could see the move on his computer screen. So the application was smart enough to see his board pieces and integrate them into the digital game that we were playing remotely together.

This sounds pretty simple, but it requires computer vision other technology to create an augmented reality experience.

The technology gives AR a visual understanding of the world, enabling applications to create AR experiences that are compelling, accessible, and useful. Through the Perceptus Platform, smartphones, tablets, and AR/VR headsets can not only see but also continuously understand 3D physical objects — all while operating on the edge to protect a user’s privacy.

This ability to dynamically identify objects, and their context, while people move unconstrained through the real world, forms the basis of a new generation of AR and mixed reality applications and experiences that could change how humans can engage with computers, Quinton said.

Effective today, the Perceptus Platform is available through licensing to interested companies.

Perceptus was developed by a team of serial entrepreneurs from Singulos Research. Led by Quinton, the team has previously founded high-tech startups that were quickly acquired by industry leaders such as Qualcomm and Tektronix. Together, the Singulos team has developed the semiconductor technologies and software at the core of most modern smartphones and mobile processors today.

I’m sure we’ve all futuristic movies or concept videos from the big tech companies. From those, you may expect that AR already provides a host of useful features, but in actuality, today’s AR lacks a visual understanding of the world around it, he said.

In a demo, he showed a video that used Perceptus to identify all of the parts necessary to put together a piece of furniture. One by one, the iPad camera identified and color-coded each piece. One piece was missing, and the camera moves in one direction and finds the missing piece.

In another demo, the iPad camera hovers over a bunch of Lego bricks and individually identifies them. You can see it identify and categorize each piece. And then it figures out what you can build with all of those pieces. The company calls this dynamic comprehension, and it works with any lighting conditions, head movements, or object rotations.

“We have the understanding of that Lego piece that you have in front of you, and we’re helping you think of ideas of things you could build,” Quinton said, and he proceeded to do that demo. “I’m going to start detecting all of these pieces. And so one of the important things to notice here is that Perceptus is understanding where these pieces are in three dimensions, even when they’re not in the camera view. So if I move the camera around, it rediscovers pieces that I already know about. It adds the sum to the inventory when it finds new ones.”

You can see some of the potential as the combination of the device and the application get smarter.

“Now Perceptus is saying you could build that item with this specific inventory,” Quinton said. “It could say I could build something, or it could say I don’t have all the right colors. It is adapting to the environment. We think that app developers can build all kinds of apps that interact with the real world.”

The Lego app could have a database of designs that it can access to determine what you can build, like a plastic duck.

“The cool thing is, it’s only going to show you things that are workable with what’s in front of you,” Quinton said.

Eventually, it would be great if Perceptus could tell you step by step how to build something. But it doesn’t do that today in the demo, Quinton said. If anyone, customers would be the ones to figure out how to build such things with their applications.

“If you’re a furniture manufacturer, of course, you already have the instructions,” he said. “So it’s a matter of having the machine learn all the different parts.”

It’s similar to the Osmo AI tech that uses a camera on an iPad to recognize items in a children’s learning application.

Quinton said that understanding objects, in real-time, is extremely challenging. In fact, that’s one reason why so much of the initial focus of the metaverse is limited to completely virtual worlds. It’s far simpler for a computer to understand the artificial world it has created. But Singulos believes that for most applications people prefer to exist in the real world.

More specifically, Perceptus’ core technology uses a completely new and proprietary approach to AI and computer vision that allows the platform to continuously identify, understand, and track objects. This technological breakthrough is what enables a new class of compelling and useful AR experiences.

Singulos also relies on existing mobile processor technology — Any modern smartphone, tablet, and goggles/glasses can deliver intelligent, physically-aware AR experiences, making AR experiences more accessible and mainstream.

And it has the ability for local control, without the cloud. Processing is done completely on the edge, like on the iPad, giving users direct control of their data. This is, of course, appealing for any company that takes privacy seriously.

The Singulos Research team is currently hosting conversations with technology companies that are interested in licensing arrangements to utilize Perceptus and incorporate the platform into their own applications and devices.

When you move an item on the chessboard, “the thing I think that’s exciting about this is this idea that you are now manipulating my reality, right?” Quinton said.

The company is talking with potential licensees who can develop their own applications. Quinton said the first customers may come out with something by the summer. He said the tech is targeted at Qualcomm’s Snapdragon processors, Apple’s M1 processor, and one of Google’s processors.

“What we need to make this work are the neural accelerators,” he said. “All of the modern mobile phones have neural accelerators, like the Apple neural engine, and the hexagon on the Qualcomm chipsets. And so that’s, that’s really what’s required. So we can be Android, we can be iOS, that’s not a problem. But it needs to be a fairly modern mobile processor.”

Over time, the tech will get better with faster and better AI hardware. The company has been around for three years and it has six people.