What Is Gemini Live and How Do You Use It?

Later in the year, Google will imbue Gemini Live with Project Astra, the computer vision tech it teased at its developer conference in May. This will allow you to use your phone’s camera app and, in real time, ask Gemini about the objects you are looking at in the real world. Imagine walking past a concert poster and asking it to store the dates in your calendar and to set up a reminder to buy tickets.

Talk to Me

Our experiences using voice assistants until this point have largely been transactional, so when I chatted with Gemini Live, I found initiating a conversation with the bot to be a little awkward. It’s a big step beyond asking Google Assistant or Alexa for the weather report, to open your blinds, or whether your dog can eat celery. You might have a follow-up here and there, but it was not built around the flow of a conversation the way Gemini Live was.

Hsiao tells me she enjoys using Gemini Live in the car on her drive home from work. She started a conversation about the Paris Olympics and about Celine Dion singing at the opening ceremony. “Can you tell me a little bit about the song she sang?” Hsiao asked. The AI responded with the song’s origin, writer, and what it meant, and after some back and forth, Hsiao discovered Celine Dion could sing in Chinese.

“I was so surprised,” she says. “But that just gives you an example of how you can find out stuff; it’s an interaction with technology that people couldn’t have before this kind of curiosity and exploration through conversation. This is just the beginning of where we’re headed with the Gemini assistant.”

In my demo, I asked Gemini what I should eat for dinner. It asked if I wanted something light and refreshing or a hearty meal. We went on, back and forth, and when Gemini suggested a shrimp dish I lied and said I was allergic to shrimp, to which it then recommended salmon. I said I didn’t have salmon. “You could always grill up some chicken breasts and toss them in a salad with grilled salad and a light vinaigrette dressing.” I asked for a recipe, and it started going through the instructions step by step. I interrupted it, but I can go back into the Gemini app to find the recipe later.