Codewhale Computer Use 0.12.0 — the agent gets its own pointer

Computer Use 0.12.0 gives the agent its own pointer on macOS. It also adds the shared-computer attach mode for Codewhale Computers and a safety floor for app scripting, irreversible clicks and consent.

• The agent never drives your cursor (macOS). Every click, hover, drag and scroll is sent to the bound app's window as a window-routed event, in both background and activate:true modes. The old route that moved the real cursor and put it back is gone: the helper refuses any request to drive your cursor (real_pointer_refused), and a helper without the window route refuses instead of falling back. activate:true still brings the app forward and gives it keyboard focus, but no longer shares your pointer. Binding receipts report shared_pointer: false and pointer_route: "window-record". In a live check on the maintainer Mac, a raw click reached the target window while 133 samples of the real cursor never came within 421 px of any agent target.
• Held drags are delivered on release. pointer down/move/up build the drag on the agent's pointer and send it to the window on up, so nothing is ever held on a real mouse button. The agent can't observe the screen mid-drag.
• Shared-computer attach mode. CODEWHALE_CU_BROWSER_ATTACH connects to a Codewhale Computer's shared Chromium instead of launching a private browser; CODEWHALE_CU_LEASE_FILE makes input tools refuse computer_busy_human_driving while a person holds the control lease; codewhale-cu-turn-hold keeps a Sprite Task alive for the turn.
• Safety floor. app_script refuses shell, Objective-C bridge and keystroke escapes and puts every scripted app through per-app consent. Clicks on pay/buy/send/transfer/delete controls need an explicit confirmation for that exact call. Consent decisions can't be batched in run_actions or replayed from a trajectory, and trajectories redact typed text and clipboard writes.

Upgrading: behaviour changes for anything that relied on activate:true moving the real cursor. Raw pointer input still needs activate:true, because the window route briefly makes the app key. Windows and Linux raw input is unchanged and still shares the desktop.

Downloads: Mac users should choose the notarized universal DMG; the ZIP is also available for installation and updates. Windows users can extract windows-x64-preview.zip and launch Codewhale Computer Use.cmd from the extracted folder; the included Node runtime needs no separate installation. Linux remains available from source and Docker. The source plugin ZIP is for plugin hosts, not a desktop installer.

The Mac app and DMG are Developer ID signed, Apple notarized (app 57e98ea7-750d-4260-a9f1-dd2aa3ab29a6, disk image 589fd26e-e2e1-4ee6-9080-c0aa5ea7d5a6), stapled, and Gatekeeper verified; all 50 packaged runtime files are byte-identical to the release commit. release.json, SHA256SUMS.txt, and release-provenance.json describe the shipped assets. Windows is unsigned and is not a production installer.

Remaining limits: continuous physical keyboard coexistence, fresh-machine permission flows, native sharing-picker integration, and physical Windows mixed-DPI qualification remain open. This release does not claim full Codex computer-use parity. Codewhale Engine ships separately and embeds its own plugin revision.

Source: release commit, three-platform CI, marketplace mirror.

Codewhale Computer Use 0.11.3 — MCP protocol conformance

Computer Use 0.11.3 is a protocol-conformance patch over v0.11.2.

• MCP conformance: the server answers resources/templates/list with an empty template list instead of -32601 method not found. It publishes a fixed skill pack and never a parameterized URI space, so an empty list is the correct answer; a host that probes the method because the server advertises resources no longer records a discovery warning at the start of every session. The capability advertisement and every existing method are unchanged, and the dispatcher still refuses genuinely unknown methods.

Users on 0.11.2 lose no capability — the warning was log noise — but they keep seeing it, because the installed bundle predates this fix. No other behaviour changes in this release; the shared-desktop, Windows preview and Linux/Docker work shipped in 0.11.2 and is carried forward unchanged.

Downloads: Mac users should choose the notarized universal DMG; the ZIP is also available for installation and updates. Windows users can extract windows-x64-preview.zip and launch Codewhale Computer Use.cmd from the extracted folder; the included Node runtime needs no separate installation. Linux remains available from source and Docker. The source plugin ZIP is for plugin hosts, not a desktop installer.

The Mac app and DMG are Developer ID signed, Apple notarized, stapled, and Gatekeeper verified. release.json, SHA256SUMS.txt, and release-provenance.json describe the shipped assets. Windows is unsigned and is not a production installer.

Remaining limits: continuous physical keyboard coexistence, fresh-machine permission flows, native sharing-picker integration, and physical Windows mixed-DPI qualification remain open. Foreground work still shares the user's input surface; use an isolated computer for uninterrupted native desktop automation. This release does not claim full Codex computer-use parity.

Codewhale Engine ships separately. Its installed acceptance was checked against its own embedded plugin revision; that is not a claim that this standalone release is already embedded in every client.

Source: canonical commit, three-platform CI, marketplace mirror.

Codewhale Computer Use 0.11.2 — shared-desktop reliability

Computer Use 0.11.2 improves shared-desktop control and adds an experimental Windows download.

• macOS beta: background typing and action routes refuse fallbacks that would borrow keyboard focus. Busy-desktop checks, per-app consent, and separate foreground consent make control more explicit.
• Windows preview: unsigned x64 ZIP with bundled Node. UI Automation targeting, geometry, control pipes, and controlled-desktop acceptance are exercised in Windows CI.
• Linux/Docker: semantic edits verify the result without replaying uncertain writes; orderly container shutdown now supports repeated display restarts.

Downloads: Mac users should choose the notarized universal DMG; the ZIP is also available for installation and updates. Windows users can extract windows-x64-preview.zip and launch Codewhale Computer Use.cmd from the extracted folder; the included Node runtime needs no separate installation. Linux remains available from source and Docker. The source plugin ZIP is for plugin hosts, not a desktop installer.

The Mac app and DMG are Developer ID signed, Apple notarized, stapled, and Gatekeeper verified. release.json, SHA256SUMS.txt, and release-provenance.json describe the shipped assets. Windows is unsigned and is not a production installer.

Remaining limits: continuous physical keyboard coexistence, fresh-machine permission flows, native sharing-picker integration, and physical Windows mixed-DPI qualification remain open. Foreground work still shares the user's input surface; use an isolated computer for uninterrupted native desktop automation. This release does not claim full Codex computer-use parity.

Codewhale Engine ships separately. Its installed 0.10.0 acceptance and pending-approval Stop fix were checked against its own embedded plugin revision; they are not a claim that this standalone release is already embedded in every client.

Source: canonical commit, three-platform CI, marketplace mirror.

Codewhale Computer Use 0.6.0

0.6.0 — window-routed background pointer and web-area traversal

• Background mouse input now reaches AppKit views without touching the
  user's cursor. Process-directed mouse events (CGEventPostToPid) never
  reach AppKit, and posting to the HID tap moves the real cursor. The
  production route addresses each event to the target window id (event fields
  0x33/0x5b/0x5c) with a window-space location
  (CGEventSetWindowLocation) and posts it as its raw event record through
  SLPSPostEventRecordTo. Measured on macOS 26.1: view-level delivery
  requires the window to be key — the window-focus record alone makes it
  only main (events arrive and are swallowed) — so the helper takes a
  momentary front-process lease with no-windows options and restores it in
  @finally, re-asserting the previous app through the Accessibility grant
  when the restore lags. Every receipt reports front_lease truthfully.
  • Coordinate left_click/double_click/triple_click/
    right_click/middle_click on a point with no pressable AX element and
    left_click_drag now deliver in background mode instead of refusing
    with shared_pointer_required. Delivery is by window id to a window
    owned by the bound app, so events cannot land on a covering window.
  • Menus survive the flow. A menu opened by a background click closes
    the moment the lease ends, so menu-opening clicks hold the lease across
    calls (state file + 15 s watchdog + restore at the next raw-input call,
    and only while the target is still frontmost). Web popup buttons report
    unpressable so they get a real click — AXPress does not open the
    native menu — and the helper polls for the menu through Chromium's
    post-activation AX rebuild. Observes poll for menu items while a menu
    lease is held.
  • Wheel scrolling uses pixel units. Chromium ignores line-unit wheel
    events entirely (measured); one notch now maps to 40 px.
  • Astral-plane typing works in occluded windows. The WindowServer
    drops key translation for covered windows, losing surrogate-pair
    graphemes (measured: "héllo wörld 日本 🐳" → "héllo wörld 日本 "). Any
    multi-unit grapheme now routes the whole keystream through the record
    channel under one lease; receipts say keyboard_delivery:"window-record".
  • Activation repaired. open_application(activate:true) uses the
    WindowServer front-process channel (options 0x200) with the AXFrontmost
    fallback, and the confirmation wait pumps the run loop — a one-shot
    helper otherwise reads a stale NSWorkspace answer for seconds.
  • set_value covers web text fields. Direct AXValue writes are
    still refused (Chromium ignores or coerces them), but the backend now
    answers with the replacement path instead of an instruction: focus,
    select-all through the window-record channel (menu key equivalents need
    a key window — new bg_key primitive), type, read-back verify.
    Receipts say strategy:"focus-type-replace" with verified from the
    control's own value — the last capability kimi-cu held over us.
  • Observation rides out Chromium's a11y rebuilds. Windows that vend
    zero content are rebuilt-tree states, not empty pages: unfiltered
    observes poll up to 2.4 s (longer while a menu lease is held) before
    returning. Filtered observes are exempt — an empty match is a legitimate
    answer.
  • Live receipts (macOS 26.1): full parity suite 28/28 tasks × 5 reps —
    background drag to a drop zone, <select> popup open + pick, native
    file-picker upload, emoji into a fully occluded window — with the real
    cursor position unchanged across every gesture and the operator's
    foreground restored.

The plugin could not see inside browser pages: get_app_state on Chrome
returned the toolbar and tab strip but never descended into AXWebArea, so
every control on the page was invisible to observe, target and verify. This
version fixes the blindness and the interaction-model friction around it,
matching the behavior kimi-cu demonstrated while keeping Codewhale's
receipts, batching, clipboard, preview and remote-computer surfaces.

• macOS observation now unlocks web content. The backend sets
  AXEnhancedUserInterface + AXManualAccessibility on target app
  elements before walking, so Chrome/Electron AXWebArea subtrees vend
  their DOM. Traversal budgets grow to depth 16 / 900 elements for
  summaries and depth 24 / 1600 for detail:"full" and query/role
  filtered observes — a filtered find can now reach deeply nested web
  controls. An AXWebArea that arrives with no descendants (page still
  populating) triggers one 200 ms re-observation before returning.
• state_id is optional on element targets. {type:"element", index}
  binds the computer's latest observation — observe, then act on the flat
  index, kimi-style. Passing state_id pins a specific earlier snapshot
  (e.g. one returned by wait_for). Live-tree revalidation,
  element_stale, state_wrong_computer and target_reacquired receipts
  are unchanged.
• Degenerate-frame refusal. Acting on a zero-size element — collapsed
  placeholder rows vended by virtualized lists (13x0, 734x1) — fails
  degenerate_frame telling the caller to scroll the row into view and
  re-observe, instead of pressing a phantom rect.
• set_value handles numeric controls honestly. AXIncrementor,
  AXSlider, AXStepper, AXValueIndicator and AXProgressIndicator
  receive an NSNumber parsed with a POSIX NSNumberFormatter; a
  non-numeric string fails before dispatch with a focus-then-type
  instruction (previously a string write could clear the control). The
  value is read back and reported as verified. Elements under
  AXWebArea refuse set_value before dispatch entirely — Chromium
  accepts AXValue sets and then ignores them, or a numeric control
  coerces the write to empty — with an instruction to focus the
  element and type instead.
• Web-area typing uses real key events. type skips the
  AXSelectedText semantic path for elements under AXWebArea
  (Chromium accepts the write and drops it) and sends process-bound
  unicode events after accessibility focus — still no pointer movement,
  still verified against the control's own value.
• The preview panel is on by default while an app is bound: a
  nonactivating mini view of the captured app window with the agent
  cursor drawn at each action's target — element-targeted actions update
  it too, not just pointer gestures. The real pointer never moves;
  preview(enabled:false) mutes it for the session.
• Source installs reuse the signed helper. When the plugin runs from
  a plain checkout (Kimi Code and other hosts' plugin dirs), the backend
  now prefers ~/Applications/Codewhale Computer Use.app's signed helper
  over compiling an unsigned one — so accessibility and screen-recording
  grants carry over instead of re-prompting or silently failing.

Live spot check on the maintainer Mac (macOS 26.1, arm64): real Chrome on
a long ChatGPT page yields ~750 elements to depth 24 including the composer
AXTextArea; the same call before the change returned ~85 browser-chrome
elements. An OCI-style create-instance form driven end-to-end through the
installed app — background, flat indices, no pointer movement — typed the
name field (verified:true), pressed the radio, refused the web
incrementor, filled the textarea and produced created:<name>. The real
OCI wizard (cloud.oracle.com/compute/instances/create) still wants its
own receipt before the docs/LIMITATIONS.md rows change.

Source suite: 260 passed, 0 failed, 15 platform skips (npm test);
Objective-C helper compiles clean in normal and CU_TEST builds.

Codewhale Computer Use 0.5.0

0.5.0 — stateful waits and persistent SSH sessions

• wait_for polls the accessibility tree until a query/role match appears (state:"present") or disappears (state:"absent"). Intermediate polls are ephemeral; the satisfying observation is rebound and returned as a fresh state_id with matched elements. Timeouts return timed_out:true receipts; cancellation, stop and computer-switch abort immediately.
• type and key accept an element target from get_app_state: the element is revalidated and accessibility-focused first, then the text or key is sent — the documented focus-then-act idiom in one call. Stale elements fail closed at stage:"focus" before any keystrokes are sent.
• recording_start accepts app_ref/window_id on macOS to record the app's window rect (fixed at start), picking the display the window lives on.
• recording_list on macOS returns .jpg/.jpeg files — screenshots saved in the system-default JPEG format were previously invisible.
• Persistent SSH sessions: agent.mjs --serve keeps one connection alive so open_application bindings and session-owned input survive between calls. Automatic one-shot fallback for older pushed agents; fail-closed on channel restart until rebind + re-observe; requestDispatched preserved on uncertain delivery.

macOS 13.5+ universal build, Developer ID signed and Apple notarized (zip submission bd1180c2-4661-401e-a5e5-4188917196d0, dmg cc2fd378-84d8-4992-b0c8-5bd9b7db9885). SHA-256 sums are in SHA256SUMS.txt and the receipt is release.json.

• .dmg — drag-to-Applications installer for humans.
• .zip — the updater's input; Check for updates… in an installed build applies this archive.

Setup and docs: https://codewhale.net/computer-use

Computer Use 0.4.0 — macOS beta

Codewhale Computer Use 0.4.0 adds the accessibility primitives a real messaging task was missing: Return/Enter from type, a filtered and paginated get_app_state, focus and get_value on observed elements, and an app click strategy that stays inside the bound app's window. The shared-desktop pointer gate is unchanged.

Install. Download the disk image, open it, and drag Codewhale Computer Use into Applications. Open it once from Applications. The setup panel walks through Accessibility and Screen Recording permissions. Apple silicon and Intel Macs on macOS 13.5 or later are supported; Node 24.21.0 is bundled, so no separate Node install or compiler is needed.

• Download and setup guide: https://codewhale.net/computer-use
• Connect it in Codewhale's plugin marketplace (Computer Use 0.4.0) so local actions go through the helper's Pause and Stop controls.
• Updates: choose Check for updates… in the app. Before installing, it verifies the archive digest, the Codewhale signing identity and the notarized Developer ID verdict, and keeps the previous app for recovery.

What changed since 0.3.1

• type treats newlines and press_enter as Return/Enter instead of inserting a literal character. key remains the named key-press tool.
• get_app_state filters (query, role), paginates (limit, offset) and truncates oversized dumps instead of eating the middle of the JSON; detail:"compact" is smaller. find_elements searches a cached state_id.
• focus and get_value act on observed elements; text-field values stay in the state dump.
• Accessibility clicks focus a field that exposes AXFocused even when it is not AXPressable.
• strategy:"app" allows a pointer event only when the point is inside the bound app's window, then restores the cursor. strategy:"event" still requires shared-desktop authorization.
• Coordinate targets accept space:"screen"; ocr_region limits OCR to a screen rect; run_actions batches up to 8 steps.
• Receipts no longer tell the model to use tools that are not in this catalog.

Verification

• Built from source commit 249ae77fad9162c2af11d5460d91b2b5b909c06c; every packaged plugin file is byte-identical to that commit.
• Developer ID signed with the hardened runtime. Apple accepted notarization submission 08f574c9-e4d7-4ff7-b7e1-431136fd4262; the stapled app passes codesign --verify --deep --strict, stapler validate, and Gatekeeper (spctl: Notarized Developer ID).
• Disk image Codewhale-Computer-Use-0.4.0-macos-universal.dmg: SHA-256 3ee12be851a9f7a2feb43fa55eea0abbd5ea9ffc83472b20ccb2d0dab7aae4d8 (88,254,027 bytes), Developer ID signed, notarized (submission 0416455e-1798-4397-b0d0-c9b1bf5cb8dd) and stapled. It contains the same notarized app.
• ZIP archive Codewhale-Computer-Use-0.4.0-macos-universal.zip: SHA-256 753565134e9fa36ac1435b64af0613d4df8195503bad56b93c80fdee0df6a637 (79,725,412 bytes). This is what Check for updates… installs. release.json and SHA256SUMS.txt cover both files.
• Source suite at that commit: 245 passed, 0 failed, 15 platform skips.

Beta scope. Qualification so far comes from one maintainer Mac. The new AX primitives are covered by the source suite; their native qualification on an installed build, a clean-machine first install with fresh permission grants, a model-driven task through an installed Codewhale Engine, and the non-admin Applications-folder update path are still being gathered as public evidence; please report anything you hit at https://github.com/Hmbown/codewhale-cu-plugin/issues. This release is the Mac helper only; it does not change the separate Codewhale Engine release.

Computer Use 0.3.1 — macOS beta

Codewhale Computer Use 0.3.1 is the first public macOS build: a notarized Mac helper with a native setup panel, whale menu-bar icon, and visible Pause and Stop controls. It operates supported app controls in the background and only takes the foreground for actions that require it.

Install. Download the disk image, open it, and drag Codewhale Computer Use into Applications. Open it once from Applications. The setup panel walks through Accessibility and Screen Recording permissions. Apple silicon and Intel Macs on macOS 13.5 or later are supported; Node 24.21.0 is bundled, so no separate Node install or compiler is needed.

• Download and setup guide: https://codewhale.net/computer-use
• Connect it in Codewhale's plugin marketplace (Computer Use 0.3.x) so local actions go through the helper's Pause and Stop controls.
• Updates: choose Check for updates… in the app. Before installing, it verifies the archive digest, the Codewhale signing identity and the notarized Developer ID verdict, and keeps the previous app for recovery.

What changed since 0.3.0

• The helper retires when its menu-bar owner disconnects, so reopening the app restores the human controls with input still stopped.
• A replacement helper's socket and run receipt survive old-session cleanup.
• The result of an update is shown after relaunch, including failed installs.
• MCP hosts receive the manifest version, and a missing registered app is explained with a repair path instead of being bypassed.
• Public plugin host eligibility is limited to macOS. Windows and Linux stay experimental, source-only backends.

Verification

• Built from source commit 9f6c39f738c0d8e8dcc93af11af5e00d19081b60; every packaged plugin file is byte-identical to that commit.
• Developer ID signed with the hardened runtime. Apple accepted notarization submission 769ff14d-ee5c-4db5-a3b9-f733c2743e6e; the stapled app passes codesign --verify --deep --strict, stapler validate, and Gatekeeper (spctl: Notarized Developer ID).
• Disk image Codewhale-Computer-Use-0.3.1-macos-universal.dmg: SHA-256 91491faa6d44b8e4b52113fea1831c07c402fdd8f673323c8125468d0d89be3a (88,246,026 bytes), Developer ID signed, notarized (submission a24464ec-f3ed-4670-b873-fcacb8a1bef3) and stapled. It contains the same notarized app.
• ZIP archive Codewhale-Computer-Use-0.3.1-macos-universal.zip: SHA-256 76752d33fff60d62b5445452e5a7f21396eb5aace6dbf632fc2a172f75e4720a (79,720,031 bytes). This is what Check for updates… installs. release.json and SHA256SUMS.txt cover both files.
• Source suite at that commit: 240 passed, 0 failed, 15 platform skips.

Beta scope. Qualification so far comes from one maintainer Mac. A clean-machine first install with fresh permission grants, a model-driven task through an installed Codewhale Engine, and the non-admin Applications-folder update path are still being gathered as public evidence; please report anything you hit at https://github.com/Hmbown/codewhale-cu-plugin/issues. This release is the Mac helper only; it does not change the separate Codewhale Engine release.