GitHub Copilot Can Click Desktop Apps. Use It Only When There Is No Better Tool.

Maestro Brief · Published by Maestro Mojo

2026-10-02 Updated 2026-10-02

2026-10-02

Maestro’s take. GitHub Copilot can now click, type, scroll, and drag inside desktop apps. That is useful for old software with no API. It is a slower, less predictable substitute when a real API, command, or MCP tool already exists.

GitHub released computer use in public preview for Copilot CLI and the GitHub Copilot app on macOS and Windows. The agent can read accessible app content and screenshots, click controls, enter text, press keys, and move through workflows across applications.

TL;DR

Copilot can now control approved desktop applications from the CLI or Copilot app.
The feature is off by default and asks before controlling an application.
An “Always allow” choice is stored locally and applies to both Copilot surfaces on that computer.
GitHub says direct tools such as APIs, terminal commands, filesystem tools, dedicated browser tools, and MCP servers are usually more structured and predictable.
Use computer control for GUI-only work. Keep destructive, financial, production, and external actions behind a human checkpoint.

What changed

Coding agents normally work through structured tools. They edit files, run commands, call APIs, or use an MCP server.

Computer use gives Copilot another option: operate the same visual interface a person would. That opens legacy desktop tools, proprietary admin apps, and other workflows that expose no better interface.

It also inherits the weaknesses of visual automation. Buttons move. Windows steal focus. Dialogs appear late. The same click can do something different after an application update.

GitHub’s documentation says computer use can select the wrong control, type in the wrong place, repeat an action, or stop when timing and window state change.

A practical developer example

Suppose your team tests a desktop application that has no automation API.

Do not say:

Open the app, test the release, fix any bad data, and publish the results.

Say:

Open the QA copy of Acme Desktop. Use the test account only. Follow the five smoke-test steps in RELEASE_CHECKLIST.md. Capture a screenshot after each step and record the visible result. Do not edit production data, approve dialogs that change account settings, send messages, or submit the release form. Stop if the screen differs from the checklist, a login prompt appears, or any destructive confirmation is shown. Return the screenshots and a pass/fail table for human review.

That prompt names the app, environment, account, procedure, evidence, forbidden actions, and stop conditions.

Do this

Prefer an API, command, filesystem tool, browser tool, or MCP server when one exists.
Start with a test environment and a low-privilege account.
Allow one application for one session while you learn how the workflow behaves.
State the exact outcome, applications, constraints, and stop conditions.
Ask for screenshots or a result table before any final action.
Keep the Stop control visible. GitHub says pressing Escape twice also interrupts an active operation.

Do not do this

Do not choose “Always allow” for password managers, banking apps, production consoles, or tools that can send messages or delete data.
Do not assume an app approval limits Copilot to one safe workflow inside that app.
Do not expose windows containing information you would not provide to Copilot as context.
Do not let the agent both enter and approve a consequential transaction.
Do not use visual clicking as a replacement for stable automation you can test and audit.

How permissions work

Computer use is disabled by default.

When Copilot asks to control an application, you can allow it for the current session, save approval for future sessions, or deny it. Saved approvals are local to that computer and shared by Copilot CLI and the Copilot app. Removing a saved approval blocks future sessions, but it does not revoke access already granted to a session that is still running.

Enterprise administrators can disable computer use with managed settings. Deny rules take precedence over saved or automatic approvals.

On macOS, the feature also needs Accessibility permission to control apps and Screen Recording permission when it needs visual context.

Why Maestro users care

A coding agent is no longer limited to code and developer tools. It can cross into the applications around the work: a legacy test runner, a desktop database client, a release checklist, or a GUI-only internal system.

That can remove a stubborn manual handoff. It can also move the agent from a predictable tool call into a screen full of ambiguous controls.

The simple routing rule is:

If a structured tool exists, use it.
If only a GUI exists, use computer control with narrow permission and visible evidence.
If the action is hard to reverse or affects another person, stop before the final click.

One thing to try

Pick one read-only GUI task you repeat every week.

Ask Copilot to collect the visible results and return screenshots. Do not let it save, submit, delete, purchase, publish, or message anyone.

If it completes the task reliably three times, consider one bounded write action with a human approval at the end.

The bottom line

Copilot computer use is a bridge to software that agents could not reach through code.

Treat it as a fallback interface, not the first choice. Give it the least access possible, require evidence, and keep the final high-impact click human.

Sources considered

Published October 2, 2026 · Tags: GitHub, Copilot, Computer Use, Developer Tools, AI Agents