computer-use-mcp v4.0.0

MCP server for Claude Code (and any MCP client): control desktop apps
through their accessibility tree plus screenshots.

What's new in v4.0.0

New pointers for the agent that move like a hand, a mouse that moves like
one too in five ways, updates that install themselves after a restart,
and the overlay kept out of screenshots on X11
(all commits).

• Six new pointers, each clicking its own way: crystal, paper, jelly,
  ice, liquid metal and orbit, 3D renders cut out with their glow (about
  65 KB each, in assets/cursors). A click moves the picture itself:

  • jelly: it squashes flat and wobbles back;
  • ice: it runs down in drips, drops fall from its points, then it
    freezes back;
  • paper: the wing opens out from the fold and folds back;
  • metal: it gives a little and drops of chrome splash out;
  • crystal: light floods through it and breaks out in colours;
  • orbit: the pearl leaves its place and races once round the ring.

  Each session picks one at random; agents working at once each get a
  different one while there are pointers to go round. The state colour
  glows behind the pointer, and the name tag takes the pointer's own
  look (clear glass with a rainbow edge, satin paper, jelly, frost,
  chrome edged in gold, dark crystal); a click shows
  for 0.7 s (was 0.45). overlay.cursor_style names one ("jelly", …),
  "classic" keeps the plain arrow, "random" is the default.

• The pointer moves like a hand (overlay.cursor_motion, on): it
  swings to where it acts in a gentle arc, leans into the move and leaves
  a trail of its own material (sparkles of colour, gold flecks, a gooey
  tail, frost, drops of chrome, a glowing streak); waiting, it breathes
  (the jelly wobbles, the paper sways, a glint runs over glass, ice and
  chrome). A drag draws its line along the way it went, in the pointer's
  material, and fades; a scroll shows arrows the way it goes.

• Keys and typing by the pointer (overlay.show_keys, on): keys
  pressed show as keycaps in the pointer's material that go down and come
  up ("Ctrl" "S"); typed text runs out a letter at a time in a bubble
  under the name tag, with a caret. Into a password field it shows dots.

• A pointer per agent (overlay.agent_cursors): { codex = "metal", "claude-code" = "jelly" } gives an agent its pointer by its MCP
  client's name or its tag, whatever cursor_style says.

• The real mouse moves like a hand (natural_mouse, on): where an
  action falls back to the mouse (a click, a drag, a scroll, a hover, the
  way to a drawing's strokes), the pointer no longer jumps there or goes
  in a straight line at an even speed, which some apps notice. It goes
  along a gentle curve to one side, quick to start and slower to settle,
  with a slight tremor, a long reach sometimes a touch past and back
  (Fitts's law timing: about 0.3 s for a short reach, 0.6 s across the
  screen); the wheel turns a notch at a time. On X11, Wayland (wlroots) and Windows the
  real pointer moves so; on macOS the app gets the same path of events
  without the user's cursor moving. Afterwards the pointer still goes
  straight back where the user left it (restore_pointer). natural_mouse = false brings back the jump.

• Five ways to move (mouse_path for the real mouse,
  overlay.cursor_path for the agent's pointer; "mixed", the default,
  picks one at random for each move): a hand's curve, a sine wave (one or
  two swings across the way, calm at both ends), a circular arc, a spring
  (an underdamped step: 6–10% past the target and back, settling) and a
  spiral in to the target (a sixth to a third of a turn). Each takes a
  hand's time and ends exactly on the target. A drag goes straight
  whatever the setting, with a hand's timing and a faint tremor: it may be
  drawing a line in a paint program.

• Updates ([update], on): five minutes after the server starts,
  then every 12 hours (shared by the servers on the computer), it asks
  GitHub for the latest release. A newer one is downloaded, checked
  against GitHub's SHA-256 for it (none, no update; only the repository's
  own release files, no pre-releases), unpacked into
  ~/.computer-use/updates/ and its program asked its version, then left
  waiting: nothing is replaced while an agent may be working. The first
  time the server starts after the computer restarts, the new version
  takes the program's place (and the package's skills and files, when it
  runs from an unpacked package or plugin) and the server goes on as it.
  install = "start" or "manual" changes when; computer-use-mcp update
  looks now, update --install puts it in now; doctor shows what waits.

• The overlay could be in the agent's screenshots on X11 without a
  compositor: the engine waited 150 ms for the overlay to say it was
  hidden, and a helper busy drawing (a glide, a fade) answered later, so
  the picture was taken with the border in it, about one time in two in
  the live test. It now waits up to a second for that answer.

Checked

• 522 tests in the library (new: every pointer keeps its tip on the spot
  through a click and while it breathes; agents at once get different
  pointers; every path style ends on the target in a hand's time without
  jumps, has its own shape, and a drag goes straight; the swing, lean and
  trail; keys, typing, scrolls and drags show and go; an agent's own
  pointer; versions, SHA-256 and CRC-32 against known values, only a newer
  checked release of the repository is taken, a zip unpacks and a bad or
  escaping one is refused, an update goes in as install says and
  replaces the program and the package), 55 in the server; clippy on Linux,
  Windows and macOS; Rust 1.88; the live Linux tests 10 of 10 with the
  natural mouse on, the overlay test 10 runs of 10 (5 of 10 before); four
  agents on one hub under Xvfb showed four different pointers. Under Xvfb
  the real pointer was traced through a click: 650 px along a curve up to
  54 px off the straight line in about 0.55 s, then back where it was;
  keycaps, the typed bubble and a drag's line were seen on the live
  overlay. Updates end to end: a build calling itself 3.9.6 downloaded
  the real v3.9.7 from GitHub (its SHA-256 matched), waited while the
  computer had not restarted, and with install = "start" put v3.9.7 and
  its package in place and went on as it; the look five minutes in found
  this version the latest. Not seen on a Windows or macOS desktop, nor on
  Wayland.

Install

Windows: download computer-use-mcp-windows-x64.zip, extract it to a
permanent folder (for example C:\Tools\computer-use-mcp), then double-click
install.cmd (or run claude mcp add --scope user computer-use -- C:\Tools\computer-use-mcp\computer-use-mcp.exe serve).
Check with claude mcp list.

As a plugin (Customize ▸ Plugins ▸ Upload local plugin): upload the full
computer-use-mcp-<os>.zip. If "Upload failed" appears and a computer-use plugin is
already installed, remove the old one first, then upload again.
Only if the full zip is refused for containing a program, use the small
computer-use-plugin-<os>.zip (about 50 KB, no program inside on purpose): first run
install.cmd -NoRegister (Windows) / ./install.sh --no-register from the full zip,
which copies the program to ~/.computer-use/bin; then upload the plugin zip.

macOS / Linux: download your platform's zip, extract it, run ./install.sh.
macOS needs Accessibility and Screen Recording permission for the app that runs Claude Code.

Every zip contains the binary, the installers, docs/CONNECT.md (Claude Code, Claude Desktop,
Codex, Cursor, VS Code, HTTP) and ready-to-copy configs in examples/.
The binaries are not code-signed, so Windows SmartScreen may ask you to confirm.

What's Changed

• v4.0.0: picture pointers, natural mouse in five ways, keys/typing/drags/scrolls by the pointer, self-updates after restart; overlay kept out of X11 screenshots by @mhrsdev in #17

Full Changelog: v3.9.7...v4.0.0