At I/O 2017 Google made a big deal out of Google Lens, which was built into the Pixel 2 and Pixel 2 XL. At I/O 2018 we learned it would be built into the Camera app with some exciting new features, such as the ability to recognise words with smart text selection.
Fast-forward to I/O 2019 and Google has another update on Lens, which is now available via the Google Assistant and within Google Photos and the Google Camera app on some Android phones. Google Go is also bringing the feature to entry-level devices elsewhere in the world. If you don't already have Google Lens on your Android phone you can download the Google Lens app free from Google Play.
Google Lens has now been used more than a billion times to help connect digital information with things in the physical world, indexing those things in the same way that Google Search indexes the millions of websites online.
A demo revealed how Google Lens would soon be able to work with menus in restaurants to show you what are the most popular options among users, along with photos of the meals. You will also be able to point your phone at the bill to help work out how much the tip should be and how to split the bill.
Soon you will also be able to point your phone at a recipe to see it come to life with a unique visual experience.
So what exactly is Google Lens?
Google Lens is a smart camera app that can read and actually understand the information within your images. We’re not talking about the image metadata, but the places, names and even Wi-Fi passwords depicted in your photos. It can then offer up intelligent ways to deal with that information. Also see: How to reverse image search
What is Google Lens is perhaps best answered with some examples: take a photo of a router’s password sticker and you’ll automatically connect to that network; snap a picture of an unknown plant and automatically identify it in Google search results; photograph foreign text for a translation; take a photo of a theatre billboard and bring up the Google Assistant to book tickets; or take a screenshot of a phone number and quickly bring it up in the dialer to call it.
Integration with Google Photos means even more information is available after the event, for example it can match your image to online data to work out what is that landmark you photographed earlier.
There are also some clever context-aware editing tools: during the demo, Google showed how its new app could digitally remove the links of a chain fence to reveal a baseball player standing behind.