Microsoft’s Copilot can browse the web with you using AI “Vision”

Microsoft is starting to test its new Copilot Vision feature today. Originally unveiled in October, Copilot Vision allows Microsoft’s AI companion to see what you see on an Edge webpage you’re browsing. You can then ask it questions about the text, images, and content you’re viewing or use it to assist you.

“When you choose to enable Copilot Vision, it sees the page you’re on, it reads along with you, and you can talk through the problem you’re facing together,” says the Copilot team in a blog post. “Browsing no longer needs to be a lonely experience with just you and all your tabs.”

It’s an entirely optional experience, and you have to explicitly grant permission for Copilot Vision to be able to read webpages in Microsoft’s Edge browser. Microsoft originally demonstrated the feature in October by showing how its AI assistant could read images from a collection on OneDrive on the web and even decipher hand-written recipes and offer up cooking tips. You could also use this feature while you’re shopping on the web to find product recommendations.

Copilot Vision is in limited testing right now, available only to Copilot Pro subscribers through Microsoft’s Copilot Labs program. It’s clear Microsoft is taking its time on this particular feature as it allows AI models to start reading things you’re seeing onscreen in a web browser, which will naturally generate privacy concerns. I’m sure the security concerns around the new Recall feature, which finally entered testing last month, played a big part in Microsoft treading carefully here.

“As we roll this out, Vision will only interact with a select set of websites to start,” says the Copilot team. “Over time, we will cautiously expand this list. It’s important to stress that Vision does not capture, store or use any data from publishers to train our models. In short, we’re prioritizing copyright, creators, and our user’s privacy and safety — and are putting them all first.”

A limited number of Copilot Pro subscribers will be able to get access to Copilot Vision today in the US as Microsoft works to listen to feedback and iterate on Copilot Vision. It plans to expand access to more Pro subscribers and websites “over time.”