Google Cloud’s Immersive Stream for XR makes possible interactive 3D and augmented reality experiences on mobile, and will soon support augmented and virtual headsets.
Back at I/O 2022 in May, Sundar Pichai credited “Google Cloud Immersive Stream” as allowing any smartphone to run the new Immersive View in Google Maps, which is just starting to launch.
Google Cloud later detailed Immersive Stream for XR as a way to put “users in an immersive, interactive, and photorealistic experience, without having to download an app.” The company believes that “mobile GPUs still struggle to create these photorealistic images,” which are needed to offer a truly immersive experience. As such, desktop GPUs are required:
The mobile phone here sends input events to our cloud service, such as the camera pose, and our cloud service then takes these camera poses and renders a photorealistic image with powerful desktop-grade GPUs, which are then streamed down to the device as video frames. The device then takes these video frames and combines them with the camera image to create an immersive photorealistic experience for this particular car model.
Today, Immersive Stream for XR is available on Android, iOS, and the desktop web. The latter platform allows you to view 3D models in Safari and Chrome, while mobile is where you find the AR mode that lets you place an object in your actual space and have it adapt to various environmental conditions, like lighting.
Google said earlier this week that it’s “actively working on an OpenXR client” to stream XR experiences to head-mounted displays (HMDs) compatible with that standard. Immersive Stream for XR on headsets will support a VR Mode and Stereo Augmented Reality Mode.
Use cases today primarily cover product demos, especially for cars as virtual showrooms. Others cover shopping, tourism, and even kitchen remodeling. Looking ahead, Google offers many more examples, including training, across different categories.
Gaming — with Stadia being Immersive Stream for Games — looks to be absent unless “Augmented Reality Sports” is meant to be interactive rather than just passive watching.
This technology and continued maturation comes as Google’s Project Iris headset is rumored to have “some graphics” rendered in the cloud and streamed to the device to avoid local “power constraints.” First-party hardware might still be faraway but it very much seems that Google is laying the groundwork for it, while improving and monetizing the experience using existing demand for non-gaming cloud streaming.
Author: Abner Li
Source: 9TO5Google
