Repository navigation
v0.7.2
Browser errors that say what to do, one coordinate space at any display scale, and choosing the window
Windows: screenshots and clicks agree at every display scale (#31)
The server was not DPI aware, so at 125% scaling a point seen at screenshot pixel (600, 550) had to be clicked at (480, 440). It now runs per-monitor DPI aware: screenshots, element bounds, clicks and window lookups all use physical pixels, and the screenshot mapping line can be used as is. Background clicks into older apps that are not DPI aware are scaled into their own coordinates. Clicks on OCR text had the same offset and are fixed too.
Windows: no clicks on controls that are off screen (#32)
A control can be listed with a position far outside its window. Google Docs' hidden "Edit document" input sits near y = -9850. Clicking one sent the real cursor there, Windows clamped it to the screen edge, and the result still said "clicked". Such a click is now refused: the error says nothing was clicked and keyboard focus did not move, and suggests type_text or set_value with element_id, which focus the control through accessibility. Coordinates that are off every display are refused the same way.
Windows: activate one specific window (#33)
get_app_statewindow lines show awindow_id, which stays the same as long as the window exists.activate_apptakeswindow_idto bring that one window forward, for an app such as Chrome where several windows share one process.activate_app,type_textandpress_keysay which window is in front or received the keys.activate_appno longer un-maximizes a maximized window.
"the extension disconnected" now says which link broke (#28)
When several MCP servers run, the first one owns the connection to Chrome and the others send their browser calls through it. If that owner had no connection from Chrome, the others got a bare the extension disconnected with nothing to act on. A browser call that cannot reach Chrome now says:
- which link is down: this server has no native host, the native host dropped during the call, the owning server exited, or the owning server itself cannot reach Chrome;
- the pid of the server that owns the connection (and its version, when it differs), and how long Chrome's native host has been away;
- that a
session_idis never the cause: sessions only keep their own tabs and share the one extension connection; - the next step: retry once (checking with
browser_snapshotbefore repeating anything unsafe), end a stale or sandboxed owner, reload the extension at chrome://extensions, or runmunim-computer-use check-native-host.
Servers of different versions keep working together.
Two instances of one app (#29)
Two Chrome profiles, for example, share a name, so get_app_state({app: "google-chrome"}) cannot know which one is meant.
list_appsshows each instance's main window title when several share a name, and ends with a note listing their pids.- On Windows and Linux the "matches several apps" error names every candidate with its pid, window count, window title and whether it is in front, so it is enough to choose from even if windows changed since
list_apps. On macOS a name still picks the frontmost instance that has windows, and the note says so.
query searches the whole outline (#30)
get_app_state with a query used to read only max_elements elements and then filter them, so a small max_elements hid matches that were on screen and reported none.
- A query now reads up to 5000 elements;
max_elementscaps only the matches printed ("N matching elements, showing the first M"). - A read that runs out of budget is reported as an incomplete search, never as a definitive empty result.
- An empty on-screen search says off-screen elements were not searched and points at
offscreen=true.
Tested on Windows 11 and macOS; cargo test passes on Windows and Linux. 32 tools (activate_app gains window_id on Windows); no extension update needed.