Google launched a computer-use feature for its Gemini Mac app on Thursday, enabling the AI assistant to control desktop applications and websites on behalf of users. The feature allows Gemini to open applications, navigate menus, fill out forms, and transfer information between local files and web platforms while users continue working on their main display. Google designed the capability to handle multi-step tasks such as extracting shipping details from PDFs, editing photo batches, and building presentations from dashboard data. The company released the feature to help users automate repetitive workflows that traditionally require manual clicking and typing. The launch positions Gemini as an active agent that operates software directly rather than simply providing text-based instructions, though Google warns that malicious content, broad application access, and imperfect backup systems can put user data at risk.
Gemini Operates in Isolated Background Window with Live Preview
The computer-use feature runs in what Google calls an isolated background window, leaving the computer's main display and cursor available for the owner. Users can watch the agent's actions through a live preview inside the Gemini chat and interrupt a task if it goes off course. Google's examples include extracting shipping details from PDF documents and entering them into online forms, editing batches of photos, and building presentations from dashboard data. The company also suggests researching prices and preparing resale listings based on pictures stored on a Mac. According to Google's support documentation, the assistant asks for permission before controlling a new app, with options to deny access, allow it once, or allow it on an ongoing basis. Users can create lists of allowed and blocked apps, monitor the virtual screen, and stop a task.
Feature Requires Apple Silicon Mac and Personal Paid Account
The feature is rolling out in English in the United States to adult users with personal paid Google accounts. It requires an Apple Silicon Mac running macOS Sequoia 15.0 or later; work and school accounts are not supported at launch. To activate it, users enable computer use in the Mac app, open Task mode, and prompt Gemini with @Computer use. Google says sensitive actions require confirmation before the agent proceeds, though an app marked "always allow" can be accessed again without another app-level prompt.
Google Warns of Prompt Injection and Irreversible Actions
Google's documentation warns that the agent can reach active sessions in desktop applications, encounter authentication codes displayed on screen, and take actions that some apps cannot reverse. The company also warns about prompt injection, an attack in which malicious instructions hidden in a document or website persuade an AI system to do something its user never requested. With permission to operate multiple apps, Gemini could encounter hostile content in one program and act on it in another. Google offers optional on-demand file backups but says they cannot always be created. Where they work, the temporary copies remain available for 24 hours or until the next prompt, and some applications may prevent their content from being backed up at all.
Users Must Enable Accessibility and Screen Recording Permissions
Users must enable macOS Accessibility and Screen & System Audio Recording permissions before Gemini can take control of desktop apps. The launch comes three days after Apple introduced tighter access controls in the macOS 27.3 beta. Those changes give users more ways to restrict powerful permissions to specific apps or folders and to require fresh approval each time access is requested.
FAQ
What did Google launch for its Gemini Mac app on Thursday?
Google launched a computer-use feature for its Gemini Mac app on Thursday that allows the AI assistant to control desktop applications and websites on behalf of users. The feature enables Gemini to open applications, navigate menus, fill out forms, and transfer information between local files and web platforms while users continue working on their main display.
What system requirements does the Gemini computer-use feature have?
The feature requires an Apple Silicon Mac running macOS Sequoia 15.0 or later. It is available in English in the United States to adult users with personal paid Google accounts. Work and school accounts are not supported at launch. Users must also enable macOS Accessibility and Screen & System Audio Recording permissions before Gemini can take control of desktop apps.
What risks does Google warn about with the computer-use feature?
Google warns that the agent can reach active sessions in desktop applications, encounter authentication codes displayed on screen, and take actions that some apps cannot reverse. The company also warns about prompt injection attacks, in which malicious instructions hidden in a document or website persuade the AI system to do something its user never requested. Google offers optional on-demand file backups but says they cannot always be created, and some applications may prevent their content from being backed up at all.