OpenComputer 0.5.0 — actions that check themselves, Take over, and a browser of the agent's own

Every action now says what changed and whether it worked, you can stop the agent or approve what matters from the activity card, and agents can browse in a Chrome of their own.

Watch the 0.5.0 demo, recorded unedited by this release's workflow on a GitHub macOS runner: an agent fills in and submits a background app while the terminal keeps the foreground, checks its own results, asks before a sensitive press, is stopped with Take over, and is refused in the terminal it runs in.

Actions that check themselves

• Diffs by default. After an action the server waits until the app's own accessibility notifications go quiet, then returns only what was added, removed, or changed. Element IDs stay valid, and "no change" now tells an action with no effect apart from one the outline can't show.
• expect and verify. Say what should happen; the result is confirmed, deviation, or unverifiable. verify checks without acting.
• Shortcuts in background apps. pressKey with via: "menu" presses the menu item that owns the shortcut; menu runs a command by path, such as File > Export as PDF….
• See what the outline can't. Screenshot marks (element IDs drawn on the picture), region zoom, on-device ocr, and maxEdge; readText for long text; waitFor; batch for up to 20 calls.
• Errors say what to do next: retrySafe, plus next and hint.
• Access levels. bundle:read and bundle:click alongside full access, and under * the terminal, editor, or chat app running the agent is read-only.
• Checkable record. release summarizes the session; the action log is hash-chained (opencomputer log --verify, --summary).

See, stop, and approve

• Activity card. The dots open on hover into a card with the agent, app, window, elapsed time, and the last three steps.
• Take over. Pauses the session until you press Resume; actions answer PAUSED_BY_USER with nothing sent.
• Approvals. OPENCOMPUTER_CONFIRM=sensitive (or a bundle ID, or all) asks on the card before presses like Send, Pay, or Delete. Allow takes a held press and is never offered to automation.
• Named ghost cursor with the agent's name tag.

Agents browse in their own Chrome

• open with app: "agent-browser" starts a second Chrome with its own profile, in the background, so agent tabs never land in your windows. com.google.Chrome always means your Chrome.
• open on the agent browser answers USER_BUSY instead of starting Chrome while a fullscreen app or game is in front.
• Screenshots fall back to the system screencapture tool when ScreenCaptureKit stalls.

Known limits: starting the agent browser can bring Chrome forward for a moment before the focus guard hands the foreground back. The demo's Take over step moves the real pointer to play the person; OpenComputer itself never posts mouse events.

Validation: this release's workflow ran the unit tests, the live fixture suites (25 and 19 assertions), and the demo on a GitHub macOS runner before publishing; the live overlay UI test passes in the Overlay screenshots workflow. The agent browser is not covered by the runner tests.

Requires macOS 14+ and Node 22+. The runtime ZIP contains an ad-hoc signed universal Apple Silicon/Intel helper. Reconnect clients after updating. Full details are in CHANGELOG.md.

OpenComputer 0.4.4 — big file panels and windows by title

Open and save panels on big folders, and steadier window choice.

• Long lists are read last. A list, table, outline, browser, or grid with more than 60 children is walked after the rest of the window. A file panel on a big folder (641 items) now still returns its OK, Cancel, and Where controls; before, they were lost to the time budget. observe also takes budgetMs (1000–15000, default 4000).
• clickPoint inside open and save panels. The host app's hit test stops at the panel's container. OpenComputer now descends to the smallest element under the point, so a pixel on Cancel presses Cancel.
• windowTitle picks a window by title, so a transient untitled window no longer shifts the index you meant.
• AXOpen joins the actions performAction accepts.

Still open: keys don't reach an open panel of a background app, and screenshots of a Chrome sheet weren't reproduced.

Requires macOS 14+ and Node 22+. The runtime ZIP contains an ad-hoc signed universal Apple Silicon/Intel helper. Running MCP servers move to the new helper by themselves (0.4.3+).

OpenComputer 0.4.3 — the MCP server keeps going after a timeout

The MCP server now survives helper timeouts and picks up rebuilt helpers.

• opencomputer serve uses ReconnectingComputer: after a TIMEOUT or a helper exit, it starts a new helper on the next request instead of answering CLOSED until the client restarts.
• It also moves to a rebuilt helper binary between requests.
• Snapshots are lost either way, so the agent sees STALE_SNAPSHOT and observes again.

Requires macOS 14+ and Node 22+. The runtime ZIP contains an ad-hoc signed universal Apple Silicon/Intel helper. Reconnect clients after updating.

OpenComputer 0.4.2 — screenshots that finish, Telegram that reads

OpenComputer now captures screenshots reliably, reads apps that hide their list rows, and covers everything since 0.3.0: background keys and clicks, focus restoration, changed-target refusals, and a per-session action log.

New in 0.4.2

• Screenshots no longer hang. ScreenCaptureKit waits forever when the helper binary lives in ~/Documents, ~/Desktop, ~/Downloads, or iCloud Drive, even with Screen Recording granted. The JavaScript client now runs a copy staged in ~/Library/Application Support/OpenComputer/helper/, and a capture that still stalls fails with CAPTURE_TIMEOUT after 12 s.
• Hidden list rows. Lists, tables, outlines, and scroll areas that expose no children (Telegram's message list, for example) are hit-tested across their visible area. What turns up is walked like any element and marked via: "hitTest". Only rows on screen are found: scroll and observe again for more.

Since 0.3.0 (0.4.0, 0.4.1)

• Keys and key-delivered typing go to the target process only. click and clickPoint press through accessibility with no synthetic mouse events. open launches apps or URLs without activating them.
• Faster batched observation with a compact outline and query search. Element IDs survive actions.
• An app that brings itself forward is sent back (focusRestored). Only activity in the same window returns USER_BUSY, after waiting up to waitMs for a pause.
• TARGET_CHANGED and STALE_ELEMENT refuse actions whose window title, tab, or page changed since the snapshot.
• One JSON Lines action log per session in ~/Library/Logs/OpenComputer. opencomputer log merges them.

Changed: app.pressKey(key, {element}) replaces pressKey(element, key). MCP observe returns text unless format: "json".

Requires macOS 14+ and Node 22+. The runtime ZIP contains an ad-hoc signed universal Apple Silicon/Intel helper. Intel is cross-compiled, not hardware-tested. Reconnect clients after updating. Full details are in CHANGELOG.md.

OpenComputer 0.3.0 — work alongside your agent

OpenComputer now checks user activity before every action on your existing Mac. It pauses for focused editing, held mouse gestures or keys, nearby pointer activity, typing, clicking, and scrolling, while allowing work in unrelated windows.

• Amber activity pill explains why an agent is waiting. Pauses report no dispatch and a retry interval; actions are never queued for surprise replay.
• Read-only checkInteraction, window/subtree observations, and explicit all-app access with OPENCOMPUTER_APPS=*.
• Existing macOS app/browser sessions stay in place. No VM, copied profiles, global input, or focus restoration.
• Thirteen JavaScript tests; live integration, activity, UI, and 28 focus/interaction checks; 21 deterministic interaction scenarios and nine motion checks.
• Claude completed with 11 calls in 30.25 seconds; Grok with 14 calls in 49.98 seconds, both with zero tool errors in those runs. Earlier protective pauses and a Grok turn-limit interruption are disclosed in the benchmark report.

Requires macOS 14+ and Node 22+. The runtime ZIP contains an ad-hoc signed universal Apple Silicon/Intel helper. Intel is cross-compiled, not hardware-tested. Reconnect clients after updating.

Apps still need usable Accessibility controls. Browser tabs share a window for conflict checks; there is no DOM adapter. Quiet input cannot prove that a person has stopped reading, and apps can activate themselves after a semantic action. The focus guard detects and contains observed changes; it is not a guarantee that every app can be automated invisibly.

OpenComputer 0.2.1 — smooth cursors and product demos

The ghost cursor now eases smoothly between controls and gently floats while idle. Animation runs at up to 60 Hz, without moving your real pointer or taking window focus.

This release also includes a product showcase made with Hyperframes from the actual native UI:

• OpenComputer-Showcase-16x9.mp4 — 1920×1080, 30 fps, 27.3 seconds, for GitHub and product pages.
• OpenComputer-Showcase-9x16.mp4 — 1080×1920, 30 fps, 27.3 seconds, for social media.
• OpenComputer-Showcase-Hyperframes.zip — both editable projects, bundled footage/fonts, provenance, and review notes.

The revised videos include ElevenLabs Eric narration, three generated sound effects, and the user-provided “night routine” by Inlirium, ducked under speech. Camera movement and text keyframes have been smoothed. Native actions and hover timing are choreographed; the footage is labelled as a scripted demonstration. The Claude/Grok benchmark result card has been removed; the independent benchmark remains in the repository.

Validation: 10 JavaScript unit tests; 13 native integration, 16 activity, 12 UI, 7 focus-guard, and 9 deterministic cursor-motion checks passed. Both revised video exports passed Hyperframes checks with zero errors or warnings and full H.264/AAC decode validation. The mix measured -16.34 LUFS and -2.67 dBTP (measured, not independently auditioned). A fixture-startup race in the UI test was fixed by selecting the test's own PID.

The macOS runtime is universal Apple Silicon/Intel, ad-hoc signed, and requires macOS 14+ plus Node 22+. Live desktop execution was tested on Apple Silicon; Intel is compiled/signature-checked. Existing focus protection remains: no raw key injection, app activation, or focus restoration by OpenComputer. Apps can still activate themselves; detected focus changes pause subsequent writes.

Watch it work · Activity behavior · Showcase details

OpenComputer 0.2.0 — activity dots, ghost cursors, focus protection

OpenComputer now shows three animated dots at the top right while an agent uses an app. Hover to expand the agent → app icons, current window/tab title, and latest action. A colored ghost cursor marks the visible target. Both panels are click-through and cannot take keyboard focus; concurrent agents get stacked indicators.

• Enabled by default. Client identity is inferred from MCP initialization or set through OPENCOMPUTER_AGENT.
• Call the new release tool when finished to remove the indicators. OPENCOMPUTER_OVERLAY=0 disables visuals while retaining focus protection.
• Raw keyboard injection and focus-setting are disabled. typeText now uses semantic selected-range replacement; pressKey is no longer advertised over MCP.
• Detected app/window/control focus changes pause further writes until reconnect. OpenComputer never restores focus or moves your real pointer. Apps can still activate themselves in response to an accessibility action; the guard cannot prevent every first or delayed transition.

Validation: 10 unit tests and 48 live native integration/activity/UI/focus assertions passed. Installed Claude Code and Grok Build each completed the same background MCP task with 11 tool calls, zero errors, correct attribution, preserved focus, and successful release. Claude: 33.93 s; Grok: 42.26 s. This is a fixture smoke test, not a general app-compatibility or model ranking claim.

The macOS ZIP includes a universal Apple Silicon/Intel helper and the JavaScript MCP server. Requires macOS 14+ and Node 22+. The helper is ad-hoc signed, not notarized. Intel is compiled and signature-checked; live desktop execution was tested on Apple Silicon.

Activity screenshots and behavior · Install · API/migration · Benchmark evidence

OpenComputer 0.1.0

OpenComputer's first public developer release provides app-scoped background computer control on macOS through a JavaScript API and a local stdio MCP server.

• Accessibility snapshots, semantic clicks/value changes, supported secondary actions, and window screenshots.
• Exact app allowlists, expiring/consumed snapshots, secure-field handling, and foreground-focus reporting.
• Claude and Grok setup guides, a disposable native fixture, and reproducible benchmarks.

Validation: 8 unit tests, 13 real macOS integration assertions, and successful Claude Code / Grok Build fixture runs. Both clients used 10 tool calls with zero errors; all mutations were performed in the background without changing focus. See the repository's benchmark report for measured timings and scope.

Download the runtime ZIP for the universal Apple Silicon/Intel helper plus JavaScript server, or the source ZIP to build yourself. Check assets against SHA256SUMS.txt.

Requires macOS 14+ and Node.js 22+ (Node is not bundled). The helper is ad-hoc signed, not Apple-notarized. Intel was cross-compiled and signature-checked, not executed on Intel hardware. This is a developer runtime rather than a graphical installer.

Keyboard input is experimental and scrolling depends on app support. Coordinate clicking/dragging, browser DOM automation, locked-screen access, Windows/Linux, and ChatGPT's private runtime/approval system are outside this release. The benchmark demonstrates the synthetic native fixture, not universal app compatibility.