Install
npx -y @harusame64/desktop-touch-mcp
Changes
Fixed
desktop_act no longer refuses elements just because their window isn't the focused
one. For elements found visually (OCR / screen recognition — anything without an
accessibility tree, such as remote-desktop sessions, games and canvas apps), the check
that decides "is this still on screen?" compared the element against whatever window
happened to be in the foreground. On a multi-monitor desktop that is usually a different
window, so the action came back ok:false with entity_outside_viewport even though the
element was plainly visible. The check now compares against the window the element was
actually discovered in, and re-running desktop_discover is no longer needed to work
around it.- A minimised or hidden window now reports why, instead of looking like a scrolling
problem. When the window an element came from is minimised or hidden, nothing is drawn
at the coordinates it was found at. desktop_act returns the new reason
origin_window_not_visible, whose recovery is focus_window(windowTitle) followed by
desktop_discover — previously this surfaced as entity_outside_viewport, whose advice
("scroll it into view") could never succeed for a minimised window.
Added
- Mouse input now works on every monitor.
mouse_click, mouse_drag, scroll,
browser_click, and the mouse route inside desktop_act can move the cursor to any
connected monitor, including monitors placed left of or above the primary one. Previously
the underlying input library silently pulled such a point into the primary monitor, so a
click meant for a second monitor reported success while clicking whatever sat at the
pulled-in position. Cursor movement now uses a built-in Windows input path that places the
cursor exactly and confirms it arrived before clicking. Movement speed settings
(DESKTOP_TOUCH_MOUSE_SPEED, the per-call speed parameter) are unchanged.
Changed
- A click that cannot be delivered now fails with a typed error instead of landing
somewhere else. CoordinateOutsideReachableBounds means the point is not on any
monitor — usually coordinates that went stale because the window moved or closed, so
re-running desktop_discover fixes it. The new CursorPlacementBlocked means the point
was fine but the pointer could not be placed there: another app is holding the cursor
(common in full-screen games), the session is not interactive right now (a disconnected or
locked remote-desktop session), or a monitor was added or removed a moment ago. Nothing is
clicked in either case, and desktop_act reports them as
reason:"coordinate_outside_reachable_bounds" and reason:"cursor_placement_blocked".
This is a behaviour change: both cases previously reported success after clicking the
wrong place. click_element invokes an element through the accessibility API without
moving the cursor and is unaffected. On an installation running without its built-in
Windows input module, mouse input falls back to the primary monitor only and the error
message says so.
