v0.5.2 — no console fallback for a busy board · untypeable text is refused whole · dsh-clawtouch 0.1.2

Fixed — with the board busy, the server opened its REPL console and wrote protocol frames into it

A Pico running the open firmware shows two serial ports: the REPL console
and the data channel. list_pico_ports marks the data one, but the server's
try-list (auto_detect_ports) appended every other likely-Pico port after
the data ports "defensively" — and with one board, that is its console. So
when another program held the data port (the ClawTouch desktop app, a serial
monitor), the server connected to the console instead and reported
"connected". Nothing is written on connect, but the first tool call wrote
protocol frames into CircuitPython's REPL, which reads what arrives as
keystrokes: a frame carries its sequence number in plain bytes, and
sequence 3 and 4 are 0x03 / 0x04 — Ctrl-C and Ctrl-D, which interrupt
and restart the firmware the other program was using. Reproduced on
Windows with the data port held open by another process.

The try-list is now the port detection marked as each board's data
channel, and nothing else. A busy board leaves the server in its no-device
state, and the next tool call says the port is busy — which is what the
message has always told the user to fix. auto_detect_port (singular)
loses its console fallback too (that branch was unreachable; the function
itself is still what the computer-use demos call).

That makes the marking itself load-bearing, so it got two fixes. Which port
is the data channel is now read from the USB interface number where the
OS reports one for every port of the board (Windows and Linux: pyserial's
location, e.g. 1-13:x.2; the console is interface 0, the data channel
2), falling back to the highest port number otherwise (macOS, whose device
names encode the interface anyway). Port numbers only usually agree:
Windows hands out COM numbers from whatever is free, so a console can sit
above its data port. And ports are grouped into boards more carefully: a
port that reports no serial number joins the board of the port at the same
USB path that does, and boards with no serial number at all are told apart
by their USB path (on every OS, macOS included) — as long as every such
port reports one; if any does not, they keep sharing one group as before,
because splitting a board would leave a lone console marked as data. Before
this, every serial-less port shared one group, which — with the padded
try-list gone — would have left all but one such board unreachable.
list_pico_ports entries gain a location field. Where the OS reports
neither interface numbers nor device names that encode them, detection
still rests on port numbers, as it always did.

Fixed — hid.type could leave text half typed

The firmware types one key per character on a US layout and stops at the
first character with no key there (Chinese, an emoji, a curly quote, "é"),
after typing everything before it — and the bridge sends text in
32-character chunks and went on to the next chunk after a failed one, so a
mixed string arrived as fragments around a gap. Now hid.type and the
type op of hid.batch check the whole text first and refuse it — nothing
typed — naming the characters, with the way round (rewrite in ASCII, or
paste from the clipboard). SerialHidBridge.type_text makes the same check
itself — the computer-use demos call it directly — raising ValueError
before any frame, and it now stops at the first chunk that is not
acknowledged. When that happens, hid.type and the batch op report chars
as the characters the device confirmed, plus unconfirmed_chars for the
chunk that failed (part of it may have been typed) — not the requested
count, which a retry would otherwise re-type on top of. The tool
description says "plain ASCII only".

Fixed — "no board" was reported as "board busy"

With no Pico plugged in at all, every tool call said the hardware was
"busy or absent" and asked the user to close the program using it. It now
says no board was found, how to plug one in, and that --mock tries the
tools without hardware; with --port given and no board detected, it says
the named port could not be opened and no board was found. device.info
gives the same reason. A board that is there but busy keeps its message.

Changed — docs: the board acts on the machine it is plugged into

Earlier versions of the README described a cross-host mode: an agent on
one machine driving another over USB HID, with nothing on the target. The
open firmware has no such command path — it reads commands from its USB CDC
data channel, the same cable that carries its keyboard and mouse — so the
board always acts on the machine it is plugged into, and clawtouch-mcp
runs there. The README (both languages), INTEGRATIONS.md, both setup
guides and the computer-use example now say so, and describe what does
work: running the agent on another machine, with clawtouch-mcp started on
the target (over SSH, for example) — keys, typing and relative moves then
work as they do locally, while absolute clicks (which read the cursor) and
hid.screenshot need the server inside the target's logged-in desktop
session (DISPLAY/XAUTHORITY on Linux; not a plain Windows OpenSSH
session).
The self-interrupt warning no longer suggests a remote target, and the
related-work paragraph no longer says this project decouples the agent from
the target machine. Also corrected: the hid.type row of the tool table
(ASCII text, not "a UTF-8 string"), a DPI-awareness comment that called the
per-monitor v1 API "v2", and the HIDAgent citation (one author).

24 new server tests: the try-list, the data port read from the interface
number, board grouping (serial-less boards, macOS locations, an unreadable
location, a port without a serial), a busy data port at startup and on lazy
retry, the typing refusal in hid.type, hid.batch and the bridge, the
bridge stopping at a failed chunk and what a failed type reports, and the
no-board messages. The regression tests among them fail when their fix is
removed; the rest pin behaviour that must not change (ASCII still types,
single-port boards still connect). 414 passed.

Fixed — dsh plugin (dsh-clawtouch 0.1.2): results that reported input which never happened, and text that stopped halfway

Three ways the plugin could tell the agent something was pressed when it
was not, or leave it guessing what was:

• mock: true reported clicks. clawtouch-mcp --mock has no device and
  answers every action with success — hid.click says clicked: true —
  and the plugin relayed that as "clicked (x, y)". The README describes
  mock as running without hardware, so it is what someone without a board
  tries first. The plugin now settles whether the server is a mock before
  it sends anything — from the --mock it started the server with, or else
  from the server's own device.info, asked once per session. Only a real
  answer counts (an object with the info block the server always sends);
  when none can be read, the tool sends nothing and says that whether the
  input would reach a device is unknown, and asks again next time instead
  of remembering a guess. A mock still receives the action, so the path is
  exercised, and the result reads "would click … — nothing was pressed:
  clawtouch-mcp is running with --mock". The same goes for sequences,
  typing, keys and scrolling; a mock server is not asked to raise a window
  either. Settling it first matters: a question asked after the input went
  out could fail and report a failure for input that happened, which
  invites the agent to send it twice.
• dryRun: true could press a title bar. It is documented as locating
  and reporting "without pressing anything", but bringing a background
  window forward is a real click on its caption (in a browser that can open
  a tab), and it happened before the dry-run check. A dry run now works
  with the window as it is, and refuses a covered one with the reason no
  raise was attempted. Every dry-run result — clicks, typing, keys,
  scrolling — says "nothing was pressed: dryRun is on". (Text that cannot
  be typed is refused in a dry run too, as it would be for real.)
• Non-ASCII text was typed halfway. The device types one key per
  character on a US layout; the first character without a key (Chinese, an
  emoji, a curly quote, "é") stops the firmware after everything before it
  was typed, and the server sent the text in 32-character chunks, going on
  to the next after a failed one (fixed server-side too, above).
  computer_type now checks the whole text
  first and refuses — before its target click, so nothing on screen
  changes — naming the characters to rewrite. The tool description says
  "plain ASCII only" up front. When the server leaves out control
  characters (on purpose: a newline must not submit a chat draft), the
  result now says how many and to send Enter or Tab with computer_key,
  instead of a bare "typed 9 characters" for a 10-character request.

The skill and the computer_click description now describe the device
plainly — a USB HID mouse and keyboard acting on this machine's screen,
with no undo — and the skill adds what the agent was missing: text is typed
on a US layout (and a Chinese input method can turn typed letters into
candidates), and "would click" / "nothing was pressed" means no input
happened. The README's hardware paragraph now says what the board does in
this plugin and how to check the locating without one: dryRun needs no
board (verified with the board's ports held by another process: the server
falls back to its no-device state and still lists windows and captures).

The typing, key and scroll tools moved into lib/actions.js so they are
tested without a host. 26 new tests in test.js: the typing check, mock
and dry-run reporting for every input too...

v0.5.1 — the frontmost window is measured, not guessed

macOS answered foreground by taking CGWindowList's first entry. That order does answer a real question — which window is topmost — just not the one the field claims: every layer-0 window an application owns is in that list, service windows included, and they sort ahead of the window the user is looking at. Measured on macOS 26, VS Code's first entry is a 1512x32 strip. Anything that trusted the flag then worked on the strip, and the symptom — a capture 32 pixels tall — reads as a calibration or HiDPI failure, which is almost impossible to trace back to its cause.

It is now a real query (NSWorkspace.frontmostApplication) scoped to that application's on-screen windows. Where the query cannot be made, the field is absent rather than false.

That last part is the theme

The same shape — something that could not be measured, reported as a definite answer — turned out to be in seven places. All are gone:

• a failed frontmost query still left foreground: false on every window
• processIdentifier() returns -1 for an application with no pid; -1 is truthy, so it looked like a real answer that matched nothing
• CGWindowListCopyWindowInfo returning NULL means the window server could not be reached — or [] turned that into "looked, and the desktop is empty"
• macOS reported minimized: false without ever asking, while include_offscreen exists precisely to include minimized windows
• find_window short-circuited on an exact title match without consulting the foreground it now had, and picked that same service strip back up
• computer_find expressed "not found" as x: -1, y: -1
• the plugin turned a failed window lookup into "no such window", and answered a request that named a window with a full-screen capture

Also

The dsh plugin's test suite had stopped running on CI. It imported index.js to test rendering, which pulls in @deepseek-ai/dsh-tools, and that job does not run npm install — it only passed on machines that happened to have the package. Rendering moved to lib/locator.js; the suite is dependency-free again.

The install line was wrong for macOS. Both READMEs asked for [screenshot] — mss + Pillow, no pyobjc — so following them produced an installation where screen.windows could never work. It now asks for [screenshot,window].

Platform support is a table, not a claim

Raising a window, occlusion measurement and the modal-input guard are Windows-only. macOS has window geometry and a real frontmost query, but neither guard. Where a measurement cannot be taken the field is absent — never filled in.

Full detail: CHANGELOG

v0.5.0 — window geometry, calibration markers, pointer-gain convergence, and the dsh plugin

Four capabilities, one theme: a wrong answer must never report success.

• screen.windows — top-level window geometry, how much of a window is actually on top, whether it accepts input at all (a modal dialog leaves a window looking perfectly normal while it discards every click), and a point where clicking raises it.
• hid.screenshot — max_width and calibration markers, so the coordinate a vision model returns maps back to the screen exactly rather than through an assumed scale.
• Absolute moves converge under pointer acceleration. Windows' "enhance pointer precision" (on by default) amplifies large deltas ~2.5x, which turned the convergence loop into an oscillation. Worst-case residual on a 7680x1440 two-monitor desktop: 328px → 5px.
• adapters/dsh/plugin — a DeepSeek Harness plugin composing the above into computer_click({ target: "the blue Send button" }). Not on npm yet.

Raising a window by clicking it

A window that is not in front cannot be worked with: a capture of its rectangle is a capture of whatever covers it. It is now raised by clicking its title bar with the real mouse, not through a focus-stealing API.

Choosing that point is the whole trick, and a real browser proved the first attempt wrong: Chrome reports its "new tab" button as a drag area (WM_NCHITTEST answering HTCAPTION), so picking the leftmost qualifying point opened a tab instead of raising the window. The caption is now scanned from the right, and a point is taken only where three consecutive probes agree — a 24px control cannot span 48px. Even then the claim is not that such a click is harmless; the window is re-read afterwards, and a re-read that cannot be confirmed is reported rather than assumed.

Two adversarial review passes

Both are recorded in the changelog. Everything they found was one category: a guard that never ran, reported as a guard that passed. Fixed: occlusion and input-state figures that were filled in as "fine" where nothing measured them, a re-read that fell open, and a window identity taken from a mutable, non-unique title.

Platform support is a table, not a claim

The raise and the occlusion/modal guards are Windows-only. macOS has window geometry but neither guard. Linux has no window list at all. Where a measurement cannot be taken it is now absent from the response — absent means not measured, never measured-and-fine.

Full detail: CHANGELOG

v0.4.6 — test-only fix (macOS CI for the 0.4.5 Retina guard)

Test-only patch. test_explicit_screen_wins_over_detection over-asserted that _detect_screen is never called for an explicit --screen; the 0.4.5 Retina guard legitimately consults it (read-only, point/pixel check) on macOS, so the test failed on macOS runners only. The test now pins a non-darwin platform; the guard's detection call is covered by TestRetinaPixelScreenGuard. No runtime change from 0.4.5.

v0.4.5 — Retina --screen guard + mouse_move magnitude docs

Rolls up the internal-only 0.4.4 (never published to PyPI) plus this release's additions.

Added

• macOS Retina --screen pixel/point guard. The OS cursor query returns CoreGraphics points, not pixels (Retina 2:1), but --screen/clamp are pixel-agnostic. A physical-pixel --screen on Retina (e.g. 2880x1800 for a 1440x900-point display) makes absolute clicks un-convergeable. The server now warns at startup on the ~2×-in-both-axes signature (distinct from a wider multi-monitor box), warning rather than rejecting.

Documented

• mouse_move int16/int8 magnitude contract. build_mouse_move packs int16 deltas; the USB HID Boot Mouse report is int8/axis and Adafruit's Mouse.move() splits |delta|>127 into successive reports. Now stated in the docstring, the firmware comment, and the protocol spec.

Fixed (folded from 0.4.4)

• Death-spiral guard on the move/glide loops (bail after 3 consecutive un-ACKed reports; flags device_nonresponsive so callers skip the dependent click).
• hid.batch stops on device non-response even with stop_on_error=false, and no longer leaks a held button on a continue-on-error failure.
• hid.click description corrected (the click is skipped when the move fails).
• CLAWTOUCH_FAKE_CURSOR env hook gated to test/mock mode only.

312 tests.

v0.4.3 — no-Pillow screenshot backend

What's new

hid.screenshot now works under hardened-runtime library-validation Python hosts — where loading Pillow's native _imaging extension is rejected (dlopen ... code signature ... different Team IDs) and the tool previously failed outright with a misleading "install [screenshot]" message.

• New mss-png backend (pure-Python, no native extension) selected via --screenshot-backend {auto,pillow,mss-png}. auto (default) uses Pillow when its _imaging loads, else falls back to mss-png. mss itself is pure-Python (ctypes → CoreGraphics, a platform framework that's exempt), so it loads where Pillow's compiled extension is blocked.
• On fallback the call succeeds (returns the image) with a friendly note that translates the dlopen error into a fix — run clawtouch-mcp from a Python without library validation, or grant the host Python the com.apple.security.cs.disable-library-validation entitlement — plus a backend field and honest scale_x/scale_y.
• New screenshot-min extra (mss only): a library-validation-safe screenshot install with no native dependency. [screenshot] (mss + Pillow, for JPEG + Retina resize) is unchanged.

Pre-release audit fixes

• mss-png now respects the 4M-pixel output cap (ceil decimation — a cap-only shrink used to round to f=1 and bypass it, returning a multi-MB PNG that could overflow the client buffer).
• Unhandled JSON-RPC notifications no longer get a spurious id:null reply (JSON-RPC 2.0 §4.1); non-dict params returns -32602 Invalid params instead of -32603; the idle-release watcher re-arms after a lazy reconnect; run_stdio logs the registered tool count.

296 tests; no behaviour change on the Pillow path. See CHANGELOG.md.

v0.4.2 — stdio robustness + doc fixes

0.4.2 — stdio loop robustness hardening + doc fixes

Fixed — stdio loop no longer crashes on valid-but-malformed JSON-RPC

An external audit found two ways a non-conforming or adversarial peer could take the whole stdio session down (a conforming MCP client never hits either — both are robustness hardening, no behavior change for real hosts):

• Valid JSON that isn't an object ([], "x", 5, true) used to raise AttributeError at dispatch()'s msg.get(...) — which runs before the parse-error guard and so slipped through to run_stdio's fatal handler, killing the session and dropping every later message. dispatch() now type-checks and returns JSON-RPC -32600 Invalid Request (id: null), keeping the connection alive — mirroring the existing -32700 handling for invalid JSON.
• Short Content-Length frame: _read_exact returns a truncated buffer on EOF; _read_framed then json.loads-ed it, so a short body whose prefix was coincidentally complete JSON could be processed as a whole message. It now verifies len(body) == Content-Length and raises a parse error (-32700) on a short read.

Regression tests added for both paths (unit + end-to-end subprocess).

Fixed — docs

• hid.click / hid.move tool descriptions: corrected stale pre-0.3.3 convergence numbers (now MOVE_MAX_ITERS=10, MOVE_TOLERANCE=5, full glide converge budget). Description-only, no behavior change.
• README.zh-CN quick-start tool count 15 → 16 (its own 14+2 breakdown already summed to 16; matches the English README and the len(tools) == 16 test).

v0.4.1 — hid.batch

hid.batch tool (run up to 10 pre-planned HID actions in one call) + back-to-back click pacing fix + absolute-click convergence recalibration. See CHANGELOG.md sections [0.4.1] / [0.4.0] / [0.3.3].

clawtouch-mcp v0.3.2

Registry packaging release.

• PyPI long-description mcp-name marker (official MCP Registry ownership verification)
• --mock Dockerfile for Glama / CI introspection

No functional or wire-protocol changes vs v0.3.1. See CHANGELOG.md for details.

This tag reconciles the GitHub release with the already-published PyPI 0.3.2.

clawtouch-mcp v0.3.1

First public release. MCP server that exposes a real USB-HID keyboard/mouse (via a Pico 2 running clawtouch-hid) as 16 tools for any MCP client / LLM agent. Cross-platform (Windows/macOS/Linux), --mock mode for no-hardware trials, screenshot off by default. This patch: stdio frames are UTF-8 on every host locale + composed tools propagate HID sub-call failures. MIT. See CHANGELOG.md.