Computer use

Requirements

• Install Desktop Control from first-launch setup or Settings → Integrations.
• Cua Driver must be available as cua-driver on PATH.
• macOS Accessibility and Screen Recording permissions must be granted for Cua Driver.
• Install the skill: ghostex computer-use install-skill.

Check machine state before acting:

which cua-driver
cua-driver status
cua-driver check_permissions '{"prompt":false}'

If the daemon is not running, start it:

open -n -g -a CuaDriver --args serve

Desktop Control readiness

Settings → Integrations shows Desktop Control as ready only when both Cua Driver is installed and the Ghostex Computer Use skill is present. The Integrations tab also reports Cua permission status separately from Ghostex's own Accessibility trust.

• Install Desktop Control — one-click installer adds Cua Driver and the $ghostex-computer-use wrapper skill.
• Grant Accessibility and Screen Recording when macOS prompts.
• Use Ghostex Computer Use for native apps; use Ghostex Browser Use for embedded browser panes.

Operating rules

• Prefer CLI calls: cua-driver <tool> '<JSON>'.
• Use the Cua Driver MCP server only when the task explicitly needs MCP mode.
• Do not register a persistent cua-driver mcp server for ordinary Ghostex Computer Use — the CLI proxies through the signed CuaDriver.app daemon.
• Do not foreground the user's apps unless explicitly asked.
• Do not use open -a, AppleScript activation, HID injection, cliclick, or menu-bar automation for background control.
• Prefer element_index from the accessibility tree over screen coordinates.

Canonical loop

• Launch or find the app with Cua Driver.
• Get window state with get_window_state.
• Choose the target by element_index from the snapshot.
• Perform one action: click, type, scroll, drag, key press, or set value.
• Re-run get_window_state and verify before continuing.

cua-driver launch_app '{"bundle_id":"com.apple.TextEdit"}'
cua-driver get_window_state '{"pid":1234,"window_id":5678}'

CLI install command

The installed skill instructs agents to load $cua-driver when available and follow its workflow exactly. Permission checks use prompt:false so state checks stay read-only.

First-launch setup

The Desktop CUA page in first-launch setup walks through one-click Desktop Control installation. You can skip it and finish later from Settings → Integrations. Skipping means agents cannot control native macOS apps until Cua Driver, the Computer Use skill, and macOS permissions are all ready.