Google is putting more of its Gemini AI into many of its products, and the next target in its sights is Photos. At its I/O developer conference today, the company’s CEO Sundar Pichai announced a feature called Ask Photos, designed to help you find specific images in your gallery by talking to Gemini.
Ask Photos will appear as a new tab at the bottom of your Google Photos app. It will roll out to One subscribers in the coming months, starting with US English. When you tap that panel, you’ll see a Gemini star icon and a bar message above the bar prompting you to “search or ask about Photos.”
According to Google, you can ask things like “show me the best photo of every national park I’ve visited,” which not only relies on GPS information, but also requires the AI to exercise some judgment. to decide which is “best”. Shimrit Ben-Yair, the company’s VP of photography, told Engadget that you’ll be able to give feedback to the AI and let it know which photos you prefer instead. “Learning is important,” Ben-Yair said.
You can also ask Photos to find the best photos from your last vacation and create captions to describe them so you can share them faster on social media. Again, if you don’t like what Gemini has to offer, you can also make changes later.
For now, you should enter your request in the “Request Photos” section. voice input is not yet supported. And as the feature rolls out, those who choose to use it will see their existing search feature “update” to ask. However, Google said that “core search features like quick access to your Face Groups or map view won’t be lost.”
The company explained that there are three parts to the Ask Photos process: “understanding your question,” “creating a response,” and “ensuring security and remembering corrections.” Although security is discussed only at the final stage, it should be baked in all the time. The company acknowledged that “the information in your photos can be deeply personal, and we take our responsibility to protect it very seriously.”
For this purpose, requests are not stored anywhere, although they are processed in the cloud (not on the device). People will not review conversations or personal information on Ask Photos except in “rare cases of abuse or harm.” Google also said it does not train “any generating AI products outside of Google Photos on this personal data, including other Gemini models and products.”
Your media continues to be protected by the same security and privacy measures that apply to your use of Google Photos. That’s a good thing, because one of the potentially more useful ways to use Ask Photos could be to get information like passport or license expiration dates from pictures you may have taken years ago. It uses Gemini’s multimodal capabilities to read text in images and find answers.
Of course, AI isn’t new to Google Photos. You’ve always been able to search for things like “credit card” or a specific friend in an app using the company’s facial and object recognition algorithms. But Gemini AI provides generative processing, so Photos can do much more than just provide pictures of specific people or things.
Other apps include getting Photos to tell you what themes you’ve used for the last few birthday parties you’ve thrown for your partner or child. Gemini AI works here to study your pictures and find out what themes you have already taken.
There are many promising use cases for Ask Photos, which is currently an experimental feature and will be “launching soon.” Like other Photos tools, it may start as a premium feature for One subscribers and Pixel owners before everyone else using the free app. There’s no official word yet on when or if that might happen.
Get all the news from Google I/O 2024 right here here!