Skip to content

v0.7.1

Choose a tag to compare

@sheehanmunim sheehanmunim released this 15 Oct 06:55
· 6 commits to main since this release

Typing alongside you, and saying why a sandboxed agent sees nothing

The agent's typing no longer lands in your window

  • Windows: clicks go in the background and leave your window in front, so type_text and press_key used to type into whatever you were typing in. Keystrokes now go to the window the agent last clicked or focused. The server waits for a pause in your typing, brings that window forward for about 10 ms, sends the text in one atomic batch, and hands the foreground back. Keys you press in the meantime are held and replayed into your window once it is back, in order. On a Windows 11 test of a user typing in one window while the agent typed in another, 22 of 22 runs came out exact.
  • type_text now sends text literally: { } ( ) are no longer read as key markup, and \r\n is one Return.
  • macOS: background clicks, scrolls, drags and hovers focused the agent's window and never gave focus back, so your typing went there or was lost. Each action now borrows focus and returns it. The exception is when the action opened a menu, including a right-click context menu, which stays open.
  • Tool descriptions no longer tell agents to call activate_app before typing, which took keyboard focus from you.

Sandboxed agents on Windows (#25)
A server started inside a sandbox can still list windows, but gets an empty accessibility tree and black screenshots, with no error. It may have a restricted or low-integrity token, run as another account, or sit on another session or desktop. The same can happen when the screen is locked. The server now detects these cases and says so:

  • get_app_state explains an empty tree instead of suggesting a screenshot.
  • screenshot and zoom flag a capture that is a single flat colour.
  • activate_app checks which window actually came forward.
  • The reason is logged once at startup.

Retrying from that context won't help; start the MCP server in your own, unsandboxed context.

Chrome extension "host not found" (#24)
Chrome reports every native-messaging lookup failure as Specified native messaging host not found. The new munim-computer-use check-native-host command shows which step fails. It looks the host up the way Chrome does (registry views and policies, the manifest's encoding, name, type and allowed origins), starts it with Chrome's own command line without disturbing a live connection, and reports whether a server is running. It also flags an install that looks fine but that Chrome can't see. That happens when the installer was run by an MSIX-packaged app, which writes to a private copy of HKCU, or by a sandbox under another Windows account. install-native-host reports the same findings under problems.

Tested on Windows 11 and macOS; cargo test passes on Windows and Linux. 32 tools; no extension update needed.