As consumers continue to await affordable augmented reality glasses, the key platforms for AR applications have been smartphones — portable devices that use their screens and cameras to create little windows into a merged real and digital world. Now a developer is using AI-backed phones to pull real-world objects into the digital realm, enabling users to instantly transfer physical items into professional apps, such as Adobe’s Photoshop, Microsoft’s Powerpoint, and Apple’s Keynote.
Currently in an invite-only beta, Cyril Diagne’s AR Copy Paste app can be held up to plants, clothes, books, magazines, and other physical objects, using a seemingly simple interface to snap a photo of each item before transferring it to an identified space in an open document. The user can simply point their Android phone or iPhone at the real object, wait a few seconds for the object to be identified and its background automatically removed using machine learning, and then point at the place on the computer screen where the digital facsimile should be pasted.
The app’s value will be instantly obvious to anyone who has created a business presentation or used an image editing tool. Including whatever you see in reality without needing to separately photograph it, mask out its background, and go through “save, switch windows, and open” stages will be a game-changer for time-strapped or photo editing-challenged professionals. It’s a rare case of machine learning, AR, and user interface simplification techniques coming together to deliver an experience that really feels like something new and magical.
Diagne was formerly the head of a Media and Interaction Design program in Switzerland, and he cofounded the Paris-based art collective Lab212. He’s currently in residency at Google’s Arts & Culture group, but AR Copy Paste appears to be his own project, rather than exclusive to Google.
If you’re interested in trying AR Copy Paste for yourself, you can request early access using a form on Diagne’s website. Android and iOS apps are available for the AR “copy” photography portion, while macOS, Windows, and Linux are all listed as supported on the computer pasting side. Productivity and creative apps such as Photoshop, InDesign, Illustrator, Figma, GIMP, Sketch, Powerpoint, and Keynote are currently listed as receivers for the pasted content. Coders interested in exploring a research prototype specific to Photoshop can find a GitHub repository here.
Author: Jeremy Horwitz.
Source: Venturebeat
