Microsoft this week enabled a new “computer use” feature in Copilot Studio, which allows AI agents to interact with websites and computer applications. Similar to the “computer use” feature of OpenAI Operator and Claude, businesses will now be able to create AI agents through Copilot Studio that will use websites and applications as task automation tools.
See also: Microsoft expands bug bounty program for Copilot (AI)
“PC usage allows agents to interact with web pages and applications by pressing buttons, selecting menus, and typing into on-screen fields,” explains Charles Lamanna, vice president of Microsoft’s Business & Industry Copilot division. “This enables agents to perform tasks even when there is no API available to directly connect to the system. If a human can use it, so can an agent.”
Copilot Studio will be able to create artificial intelligence agents that will automate data entry, conduct market research, or even process invoices via computer. According to Microsoft, this new tool will also detect changes in buttons or application and website screens, continuing its operation without errors or interruptions.
See also: Microsoft launches Copilot Chat for businesses
Microsoft added a similar feature, called Actions, to the consumer version of Copilot earlier this month. Actions can run in the background while the user works on other tasks. Copilot can now make restaurant reservations, buy event tickets, or make purchases from online stores. However, the Actions experience is currently limited to a select few partners, while Copilot Studio seems to offer more freedom, with the ability to work with more websites and apps.
One interesting thing is that Copilot Studio is moving in a direction where AI agents can function almost like digital assistants with human-like navigational abilities. That is, they are no longer limited to data from APIs or pre-defined scripts — they can “see” and interact with the user environment just as a human would, by clicking, typing, or selecting from a menu.
See also: Microsoft 365 Copilot brings enhanced AI capabilities
This opens the way for broader automation in environments that were not designed for AI, e.g., legacy applications without modern APIs or websites with frequent UI changes. Essentially, this technology bridges the gap between old and new systems.
Source: theverge
