v0.4.0 — verified outcomes

Verifier-first reliability and large-tree perception. Every claim in this release was certified by a live four-suite battery against the installed release bundle — 12/12 truth cases with zero silent false-successes, run while the user was actively using the machine (battery report). Tests grew 184 → 337.

Highlights

• Verified action outcomes. Every mutating tool does a read → act → re-read of its target and classifies whether the effect actually happened (success / unsupported / effect_not_verified / verifier_ambiguous), surfaced in _meta with a failure_domain and a plain-language verdict. A button that lies about accepting a click is reported honestly — not as success. Always on (~3% median latency, measured).
• Multi-strategy AX action chains. Clicks that can't land as a plain press walk a verified chain (confirm → open → pick → selection relay → child/ancestor press); the winning rung is reported as chain_rung.
• Background-reliable scrolling. Containers are ranked, and delivery drives the container's own AX mechanisms first — including setting the scroll bar's value, the one mechanism that actually moves an NSScrollView in the background — before falling back to synthetic wheel events.
• Dense-collection viewport windowing + skeleton: true. Virtualized lists/tables are windowed to the on-screen slice instead of a blind first-N prefix; find and skill replay still reach any row (verified live on row 400 of a 500-row table).
• Window-motion verification. Offscreen moves are rejected before any write; resizes settle-poll animated frames and report the honest applied frame when the app clamps.
• Structured errors. [CODE]-prefixed failures with machine-readable code and a one-line recovery hint in _meta.
• Delivery telemetry. delivery_tier, fallback_reasons, and ui_changed on every action.
• Visible-range read_text and web-area rich text rendered to markdown.
• ComputerUseFixture — a truth-suite app (honest/liar/disabled buttons, keystroke echo, 500-row table, web pane) that backs the release battery; launches in the background by default.
• Agent-cursor z-order rules. The overlay cursor shows whenever the target window is visible (split view, second display, unfocused) and clips only when the frontmost app's window genuinely covers the target. COMPUTER_USE_MCP_CURSOR_TOPMOST=1 forces always-on-top.
• Teach-mode durability. Text-entry anchors by role + tree path (labels churn with content); run_skill cold-launches a closed target app in the background.

Fixed

• Continuation leak that could hang daemon-less calls when a wedged replayd never answered a capture handshake.
• Agent cursor invisible over visible-but-unfocused targets on fullscreen / cross-Space setups.
• Stale-identity false negatives on text-entry elements.

Full details in the CHANGELOG.

v0.3.0

computer-use-mcp v0.3.0

This release turns computer-use-mcp from a per-action driver into a system that can
learn a task once and replay it at engine speed — the agent (or the user, via a
recorder) demonstrates a task, and it is saved as a named, self-healing skill. Around
that, the release hardens reliability (stable element ids with state diffs, a batch
tool, fresher app/display enumeration, a newest-build-wins daemon handover), tightens
safety and coexistence (interference yield, browser URL policy, screen-lock pause,
sleep assertion), and improves what the agent can see and what the user can watch.

Teach / replay skills

• Save, run, list, get, delete. New save_skill / run_skill / list_skills /
  get_skill / delete_skill tools capture a task once and replay it as a named,
  parameterized skill. Element anchors are frozen into durable locators (role + label +
  tree path) that re-resolve on each run, so a skill survives app restarts and replays
  at engine speed with no model in the loop.
• Recorder teach mode. record_skill_start / record_skill_stop capture a task the
  agent performs or that the user demonstrates with the recorder, then save it as a
  skill draft.
• Self-healing locators. A resolved-but-moved element self-heals its saved path on
  replay, so ordinary UI drift doesn't break a skill.
• Resume. A run can restart from a given step (start_at_step) after a failing step
  is fixed and re-saved, instead of re-running from the top.
• Extract steps. read_text steps return their text in the run result, so a skill
  can pull data out as it replays.
• Per-step safety and assertions. Each replayed step passes the same per-step safety
  gates as a live action, and steps can assert their effect (expect, in wait_for
  terms).
• Repair reports. A step that no longer resolves stops the run with a report naming
  the nearest candidate elements, so the failing step can be fixed and re-saved.

Reliability

• Element-id stability + state diffs. Actions return state diffs; elements that
  survive a UI change keep their ids, so ids the agent already holds stay valid instead
  of being invalidated on every action.
• Batch tool. batch runs a short, predictable action sequence in one round-trip,
  stopping at the first failure.
• Fresh enumeration fixes. Running apps are enumerated freshly rather than from an
  NSWorkspace snapshot; the frontmost app is read from the window server (with an
  lsappinfo path in live eval) rather than NSWorkspace; and display scale is derived
  from CoreGraphics rather than NSScreen.
• Newest-build-wins daemon handover. The daemon retires on binary change, not just on
  a version-string change, so a freshly built engine takes over cleanly.
• Chronic CI build fix. Daemon wire types were marked Sendable to fix the recurring
  CI build failure, a strong actor reference is bound before reader-thread Tasks, and
  the full build error is now surfaced in CI.

Safety & coexistence

• Interference yield. The server yields to real hardware input: when the user is
  working in the target app (or an action uses the global cursor) and hardware input was
  seen recently, the call returns a recoverable error instead of interleaving with the
  user (interference_idle_seconds).
• Browser URL policy. Browser actions are gated server-side on the current page URL —
  url_deny patterns block outright (confirm does not override) and url_confirm
  patterns (plus payment-page defaults) require confirm.
• Screen-lock pause. Mutating tools pause with a recoverable error while the screen is
  locked, resuming when the user unlocks.
• Sleep assertion. The Mac is kept from idle-sleeping while tool calls are flowing.
• Focus give-back on cold launch. After a cold app launch steals focus, focus is
  returned to the app the user was in.

Visibility

• Click pulse. The overlay shows a pulse at each click location.
• Agent-working chip on all displays. The "Agent working" status chip is shown on
  every display, not just the primary one.
• Recording indicator. Teach-mode recording surfaces an on-screen indicator.

Perception

• Opaque-canvas OCR hint. Rich-shell windows that draw their own opaque canvas are
  detected and hinted for OCR (ocr: true).
• Unlabeled-control identifiers. Unlabeled native controls get a richer identity in
  the outline so the agent can address them.
• Inactive-Space capture. Screenshots can capture windows on inactive Spaces.
• include_screenshot on get_app_state. get_app_state honors
  include_screenshot, with a documented compatibility matrix and detail-selection unit
  tests.

Ops

• One-command deploy. scripts/deploy_app_bundle.py does a release build, bundles the
  .app, installs to ~/Applications, and performs the daemon handover;
  --check exits non-zero when the installed bundle is stale.
  build_app_bundle.py gained --configuration, --identity, and --install so release
  binaries can be wrapped and signed.
• Telemetry in health_report. Per-tool funnel counters are surfaced via
  health_report.
• CI. A duplicate bare build step was dropped, and regression coverage was added for
  the pid-scan count fix.

Upgrading

Run python3 scripts/deploy_app_bundle.py to build the release, wrap and install the
.app bundle, and hand the daemon over to the new build. Already-running MCP sessions
keep the old tool list until they are restarted — restart your MCP client (or its
serve shims) to pick up the new tools.