Google has started rolling out new AI features to Gemini Live that let it “see” your screen or through your smartphone camera and answer questions about either in real-time, Google spokesperson Alex Joseph has confirmed in an email to The Verge. The features come nearly a year after Google first demonstrated the “Project Astra” work that powers them.
A Reddit user said the feature showed up on their Xiaomi phone, as spotted by 9to5Google. Today, the same user posted the video below demonstrating Gemini’s new screen-reading ability. It’s one of the two features Google said in early March would “start rolling out to Gemini Advanced Subscribers as part of the Google One AI Premium plan” later in the month.
The other Astra capability rolling out now is live video, which lets Gemini interpret the feed from your smartphone camera in real-time and answer questions about it. In the demonstration video below that Google published this month, a person uses the feature to ask Gemini for help deciding on a paint color to use for their freshly-glazed pottery.
