Microsoft’s new Copilot Vision feature that can “see what you see, and hear what you hear” while you navigate the internet is finally being made available, though only to a limited number of Copilot Pro subscribers in the U.S.
“Starting today, we are introducing an experience where – with your permission – Copilot can now understand the full context of what you’re doing online,” according to a Microsoft blog post. “When you choose to enable Copilot Vision, it sees the page you’re on, it reads along with you, and you can talk through the problem you’re facing together.”
The feature was first teased in October alongside the debut of Copilot Voice, Microsoft’s answer to ChatGPT’s Advanced Voice Mode and Google’s Gemini Live. It is currently available as a preview to Pro subscribers through Copilot Labs and available exclusively on Microsoft’s Edge browser.
To help alleviate user concerns (and distance the new feature from the company’s troubled Recall launch), Vision will have to be specifically activated whenever the user wants to employ it and will display a persistent icon (akin to your webcam’s On light) until the user turns the feature back off.
While active, the AI assistant will “scan, analyze, and offer insights based on what it sees.” The system can suggest next steps to take, answer questions about the displayed content, navigate to other parts of the site, and assist with various online tasks.
Having Copilot help you surf the web is only the start to Microsoft’s AI assistant plans. In January, the company is expected to release the first of its next-generation AI agents, which will autonomously analyze available data to perform tasks on the user’s behalf.
“Copilot will ultimately be able to act on your behalf, smoothing life’s complexities and giving you more time to focus on what matters to you,” Mustafa Suleyman, executive vice president and CEO of Microsoft AI, wrote in October. “It’ll be an advocate for you in many of life’s most important moments. It’ll accompany you to that doctor’s appointment, take notes and follow up at the right time. It’ll share the load of planning and preparing for your child’s birthday party. And it’ll be there at the end of the day to help you think through a tricky life decision.”
Read the full article here
