Google Lens screen search coming to Assistant on Android

Google is following this week’s announcement of Bard with a slew of new Lens capabilities, including “Search screen” on Android and wider Multisearch availability. 

To start with, Google Lens is now used over 10 billion times per month. This is up from 8 billion/month in September.

“Search screen” is a proper replacement for Assistant’s “What’s on my screen?” functionality, which dates back to Google Now on Tap. Today, evoking Google Assistant might result in a panel showing shortcuts for “Lens” and “Read.” The former basically takes a screenshot and sends it to Lens for analysis, but it does not consistently appear in every app/instance. 

Search screen will always be available, and Google promotes it as letting you “search what you see in photos or videos across the websites and apps you know and love, like messaging and video apps — without having to leave the app or experience.”

• “Say your friend sends you a message with a video of them exploring Paris. If you want to learn more about the landmark you spot in the background, you can simply long-press the power or home button on your Android phone (which invokes your Google Assistant) and then tap “search screen.” Lens identifies it as Palais du Luxembourg — and you can click to learn more.”
• “Or perhaps you notice an interesting-looking chair in the video, and want to find where to get it or chairs that look like it. Simply long-press the power or home button on your phone (which invokes your Google Assistant) and then tap “search screen” to find options.”

Google is rolling out Search screen globally over the coming months.

Meanwhile, Multisearch, which lets you take a picture and add a text query to it, is now globally available in all Google Lens languages and countries. The ability to append “near me” to a Multisearch is getting similar global availability in the coming months. 

Multisearch is also coming to Google Images and any picture that appears in Search results on mobile web:

For example, you might be searching for “modern living room ideas” and see a coffee table that you love, but you’d prefer it in another shape — say, a rectangle instead of a circle. You’ll be able to use multisearch to add the text “rectangle” to find the style you’re looking for.