After several months of previews and select releases, Microsoft’s Copilot Vision has officially launched for Edge users in the U.S. This exciting new AI feature is designed to observe your screen while you browse, allowing you to ask questions about your activities and receive relevant context-based answers. Currently, however, it is only compatible with nine specific websites.
The list of these nine websites appears somewhat eclectic. While having Amazon is logical, the inclusion of Geoguessr is puzzling, considering the site’s primary purpose is to guess locations without assistance. The complete list of compatible sites includes:
- Wikipedia
- Tripadvisor
- Williams Sonoma
- Amazon
- Target
- Wayfair
- Food & Wine
- OpenTable
- Geoguessr
Mustafa Suleyman, Microsoft AI’s CEO, recently announced the launch on Bluesky and highlighted several of his favorite use cases.
Typically, when querying Copilot, users need to manually enter paragraphs of context, which can be time-consuming and challenging, especially when inquiring about unfamiliar topics.
With Copilot Vision, rather than attempting to articulate what you’re viewing or discussing, the AI can directly observe your screen.
For instance, according to Suleyman, if you search for “breathable sheets” on Amazon, you could ask Copilot whether any of the results consist of suitable materials. It can identify the correct ones or provide examples of breathable fabrics to look up.
On the Food & Wine website, Copilot can assist you while cooking by responding to your questions and narrating portions of recipes, as this interaction is primarily voice-driven. You can speak to the AI, and it responds in kind.
The Copilot Vision page indicates that you can also type your inquiries and receive written answers.
Microsoft is taking a cautious approach with this feature, likely to avoid another backlash similar to what occurred with Recall. The limited selection of compatible websites stems from potential copyright concerns, and the company emphasizes that this feature is opt-in. It does not record your screen, is only active when you turn it on, and deletes data after each session concludes.
If you’re interested in exploring this new feature, you can set it up and watch a tutorial on the Microsoft website.
