It’s the moment you’ve been waiting for all year. Google I/O main day. Google kicks off its developer conference every year with a rapid stream of announcements, including lots of revelations about the latest things it’s working on. Brian: has already kicked us off by sharing what we expect.
We know you don’t always have time to watch the entire two-hour presentation today, so we’re embracing it and will provide quick hits of key news as they’re announced, all in an easily accessible format. digestible, easy to digest list. Let’s go!
Google Play
Google Play is getting some attention with a new discovery feature for apps, new ways to acquire users, updates to Play Points, and other improvements to developer tools like the Google Play SDK Console and Play Integrity API, among others. of other things.
Of particular interest to developers is something called the Engage SDK, which will allow app creators to present their content to users in a full-screen, immersive experience that is personalized to the individual user. Google says this is not a surface that users can see at this time. read more
Detection of scams in calls
On Tuesday, Google previewed a feature it believes will alert users to potential scams during a call.
The feature, which will be built into a future version of Android, uses Gemini Nano, the smallest version of Google’s Generative AI offering that can run entirely on a device. The system effectively listens in real time for “conversational patterns commonly associated with fraud.”
Google cites an example of someone claiming to be a “bank representative.” Common scam tactics like password requests and gift cards will also trigger the system. These are all pretty well understood ways to get money out of you, but many people in the world are still vulnerable to these types of scams. After leaving, a notification will appear that the user may become a victim of unpleasant characters. read more
Ask for Photos
Google Photos is getting AI-infused with the launch of a pilot feature, Ask Photos, powered by Google’s Gemini AI model. The new addition, coming later this summer, will allow users to search their Google Photos collection using natural language queries powered by AI insights into their photos’ content and other metadata.
While users could search for specific people, places or things in their photos thanks to natural language processing, AI upgrades will make finding the right content more intuitive and less of a manual search process.
And the pattern was beautiful too. Who doesn’t love a tiger-stuffed Golden Retriever duo called Golden Stripes? read more
All about Gemini
Gemini 1.5 ProAnother update to generative AI is that Gemini can now analyze longer documents, codebases, videos and audio recordings than before.
A private preview of a new version of the company’s current flagship, the Gemini 1.5 Pro, revealed that it can accept up to 2 million tokens. That’s double the previous maximum amount. At that level, the new version of the Gemini 1.5 Pro supports the largest input of any commercially available model. read more
Gemini Live: The company previewed a new Gemini experience called Gemini Live, which allows users to have “deep” voice conversations with Gemini on their smartphones. Users can interrupt Gemini while the chatbot is speaking to ask clarifying questions, and it will adapt to their speech patterns in real time. And Gemini can see and react to users’ surroundings, either through photos or videos captured by their smartphone cameras.
At first glance, Live doesn’t seem like a drastic upgrade over existing technology. But Google claims to use newer techniques from the generative AI field to deliver better, less error-prone image analysis, and combines these techniques with an enhanced speech engine for more consistent, emotionally expressive and realistic multi-turn dialogue. read more
Twin Nano. Now for a little announcement. Google is also building Gemini Nano, the smallest of its AI models, directly into the Chrome desktop client starting with Chrome 126. This, the company says, will enable developers to use the on-device model to power their own AI capabilities. Google itself plans to use this new feature to power features such as Workspace Lab’s existing “help me write” tool in Gmail. read more
Gemini on AndroidGoogle’s Gemini on Android, the AI replacement for Google Assistant, will soon take advantage of its deep integration with Android’s mobile operating system and Google apps. Users will be able to drag and drop AI-generated images directly into their Gmail, Google Messages and other apps. Meanwhile, YouTube users will be able to tap “Ask this video” to find specific information from that YouTube video, Google says. read more
Gemini on Google Maps. Gemini features are coming to the Google Maps platform for developers, starting with the Places API. Developers can display generative AI summaries in their own apps and websites. Summaries are created based on Gemini’s analysis of insights from the Google Maps community of over 300 million contributors. What is better? Developers will no longer have to write their own custom descriptions of locations. read more
Tensor Processing Units get a performance boost
Google has unveiled its next generation of its Tensor Processing Units (TPU) AI chips, the sixth to be exact. Called Trillium, they will launch later this year. If you remember, it’s a tradition at I/O to announce the next generation of TPUs, even though the chips aren’t released until the end of the year.
These new TPUs will have a 4.7x increase in computing performance per chip compared to the fifth generation. Perhaps even more importantly, however, Trillium represents the third generation of SparseCore, which Google describes as “a specialized accelerator for processing the very large embeddings common in advanced classification and recommendation workloads.” read more
AI in search
Google is adding more AI to its searches, allaying suspicions that the company is losing market share to rivals such as ChatGPT and Perplexity. It provides AI-powered reviews for US users. Additionally, the company is also looking to use Gemini as an agent for things like travel planning. read more
Google plans to use generative AI to organize the entire search results page for certain search results. This is in addition to the existing AI Overview feature, which creates a short snippet with comprehensive information about the topic you’re searching for. The AI Overview feature becomes generally available on Tuesday after a hiatus in Google’s AI Labs program. read more
An upgrade to generative AI
Google has unveiled Imagen 3, the latest in the tech giant’s Imagen family of generative AI models.
Demis Hassabis, head of DeepMind, Google’s AI research arm, says Imagen 3 more accurately understands the text prompts it translates into images than its predecessor, Imagen 2, and more ” creative and detailed” in its generations. In addition, the model produces fewer “distracting artifacts” and errors, he said.
“This [also] Our best model yet for text reproduction, which was a challenge for image generation models,” Hassabis added. read more
Gemma 2 updates
Gemma 2, the next generation of Google’s Gemma models, will launch in June with a 27 billion parameter model. read more
Project IDX
Project IDX, the company’s next-generation, AI-powered browser-based development environment, is now in open beta. This update integrates Google Maps Platform into an IDE that helps add geolocation features to its apps, and integrates with Chrome Dev Tools and Lighthouse to debug apps. Soon, Google will also enable applications to be deployed on Cloud Run, Google Cloud’s serverless platform for running front-end and back-end services. read more
Wow!
Google is taking a shot at OpenAI’s Sora with Veo, an AI model that can generate 1080p videos around a minute long at a text prompt. Veo can capture a variety of visual and cinematic styles, including landscape and time-lapse shots, as well as make edits and adjustments to already created footage.
It also builds on Google’s initial commercial work in video creation, previewed in April, which used the company’s Imagen 2 family of models to create scrolling videos. read more
Scope to search
The AI-powered Circle to Search feature, which allows Android users to get instant answers using gestures like circling, will now be able to solve more complex problems with psychic and math word problems. It’s designed to make working with Google Search more natural from anywhere on your phone by performing specific actions like circling, highlighting, doodling, or tapping. Yes, and even better, help kids with homework right from supported Android phones and tablets. read more
Firebase Genkit:
There’s a new add-on to the Firebase platform called Firebase Genkit that aims to make it easier for developers to build AI-powered apps in JavaScript/TypeScript, soon with Go support. It’s an open source framework using the Apache 2.0 license that enables developers to quickly build AI into new and existing applications.
Some of the Genkit use cases the company highlighted on Tuesday included many of the standard GenAI use cases: content generation and summarization, text translation and image generation. read more
Pixel 8a:
Google couldn’t wait until I/O to show off the latest addition to the Pixel lineup and announced the new Pixel 8a last week. The phone starts at $499 and ships on Tuesday. The updates are also what we have come to expect from these updates. At the top of the list is the addition of the Tensor G3 chip. read more
Pixel Slate:
Google’s Pixel Tablet, dubbed the Slate, is now available. If you remember, Brian reviewed the Pixel Tablet around the same time last year, and all he talked about was the basics. Interestingly, the tablet is available without it. read more
We’ll be updating this post throughout the day…