How You Search For Almost Anything On The Internet
Exploring Google Lens and MUM Technology
Google Lens may change the way you search for obscure or difficult to find items, increase enjoyment while traveling, or even help with homework. The app came pre-installed on my android phone, promising a glimpse into futuristic technology. Over the holidays, I finally had a chance to explore all the features. Google Lens uses AI visual recognition powered by neural networks to recognize images saved on your phone, screenshots or live images you snap. Using artificial intelligence pre-trained with text and images, internet searches are more intuitive. You can add text to an image, making the search more relevant. Here’s an example.
We have a vacation coming up, and my husband needs a wardrobe update. He’s impossible to shop for, and doesn’t want to try anything on. One night while watching a movie, he says, “I like what he’s wearing.” I hit pause, snapped a pic through the Google Lens app, and up pops a page of clothes, with exact matches or very close, and at a range of price points. I can manipulate the cropping to focus on the top, pants or shoes, even the sunglasses. For the pants, I added the word “mens” to further refine the results only to men’s pants. If you are wondering if it works taking a pic of a live person, yes, it does. I took a photo of a friend’s shoes while he was wearing them and the app performed wonderfully, coming up with several exact matches, and where to buy them.
If you have an article of clothing that you like, or find a photo online, you can type in “find matching accessories like shoes, a purse or scarf” and create an entire ensemble.
Clothes and shopping are just one thing Google Lens does better than any standard internet search.
So far, we have found parts for a broken wall safe, added a contact from a business card with links to their professional profiles, snapped a photo of an event poster and added it to our shared calendar, and scanned barcodes of items to see where to buy them. There is even a menu tab to get help with homework, which would have been handy in calculus class, or a translate button to quickly convert that foreign language sign to English when traveling. But probably the most fun is using the app outdoors.
There are a few plants on the outer edge of our small farm I haven’t identified. Snap, snap, now I know! I also have two types of a similar plant growing in one pot, and could not tell them apart, and Google searches were no help. The only difference between them is that one has skinnier leaves. Photographing them and then cropping each stalk individually helped confirm which one is which.
You can also use Lens to identify natural elements like a mountain in the distance. You may have to crop the most recognizable parts of the image, like the distinct features of the summit, to create search data points. This page on tracelabs.org shows how to do this from a photo, and I was able to duplicate the search following the steps.
Like getting a new toy for Christmas, I’m having a hard time not using Lens! The best part is, it’s free, as long as you don’t mind giving up more privacy access to Google.
As an Amazon Associate, Arvig earns from qualifying purchases.
The technology behind Google Lens
In May 2021, Google introduced a Multitask Unified Model (MUM) as a 1000 times more powerful evolution of the previous search algorithm Bidirectional Encoder Representations from Transformers (BERT). After MUM started rolling out in Google Lens later that year, it clearly exceeded earlier natural language processing. It is now incorporated into Android phones as part of the Google ecosystem. You can use it on an iPhone and iPad if you download the Google Lens app.
MUM combines several technologies to go beyond normal search results to answer complex queries. Google fed a massive amount of data into MUM to provide a deep understanding of world knowledge, including proficiency in 75 languages simultaneously. MUM can not only combine information gleaned from a photo and text, but looks at all data incorporated behind the media too. The end goal is a complete global knowledge database, which is wonderful and terrifying at the same time.
How to access Google Lens
Even if your Android came preloaded with Google Lens activated on other Google apps, downloading and accessing the tool through the app is the recommended approach for both Androids and iPhones. Here’s how to load and use it.
- Download and open the Google Lens app from Play Store or iTunes.
- Alternately, you can ask Google Assistant to activate Google Lens on your Android phone.
- Accept Permissions for access to your camera and saved photos.
You can find out more details about using Google Lens here.
Give Lens a try!
Whether you’re doing some shopping, looking up an obscure item, embarking on vacation, or exploring closer to home, Google Lens can help learn more about the world around you. The search results are the most groundbreaking to date.