Microsoft has unveiled a groundbreaking feature, dubbed “computer use,” that empowers AI agents to autonomously handle data entry, market research, and various other tasks across desktop and browser apps. The new functionality, which is currently available in a research preview for select users, is accessible through the Copilot Studio platform and is set to revolutionize how businesses automate routine operations.
The computer use feature enables AI agents to navigate applications seamlessly, performing tasks like clicking buttons, selecting data, and filling out forms—all without the need for manual input. Whether working within popular desktop apps or browser-based platforms like Microsoft Edge, Google Chrome, or Firefox, the AI agents mimic the actions of a human user, making them versatile tools for a wide range of tasks.
One of the primary use cases for the feature is automating the entry of large volumes of data into centralized systems, particularly in industries that rely heavily on processing data from multiple sources. This can significantly reduce the manual effort involved, minimizing human error and improving accuracy. Microsoft envisions the technology also being valuable in tasks such as market research and invoice processing, where repetitive activities can be time-consuming and prone to mistakes.
What sets Microsoft’s computer use feature apart is its advanced reasoning capabilities. Unlike other AI agent tools, which may require human intervention when encountering obstacles such as CAPTCHA challenges, Microsoft’s AI agents are designed to handle these hurdles independently. If buttons or screens change, the agents adjust without disrupting the workflow. This level of adaptability ensures that the agents can continue functioning even as environments evolve.
Users also have transparency into the actions taken by the AI agents, with the option to view a history of completed tasks, including screenshots and the reasoning steps followed to resolve issues. This added visibility is crucial for maintaining control over automated processes.
Overall, Microsoft’s new computer use feature is a significant leap forward in automation, offering businesses an efficient, error-reducing solution for a variety of complex tasks.
