claude-workman is an open-source MCP server that gives an AI assistant human-mode control of a Linux desktop — it takes a screenshot, reads the accessibility tree, then moves the mouse, clicks, types, and switches windows. It works with Claude, Claude Code, and any other MCP client.
Named after the Walkman: a small, portable thing that just plays. Point it at a display and it works.
pip install claude-workman
claude mcp add --scope user workman -- python -m workman.serverMost desktop automation guesses at pixel coordinates and breaks the moment a layout shifts, a font renders differently, or a window moves. claude-workman fuses two signals on every turn — a screenshot and the AT-SPI2 accessibility tree — so an action can be addressed semantically:
"click the push button named Continue"
…and land on that button's center, wherever it moved to. This is the same principle behind
macOS AXUIElement-based computer use, ported to Linux via AT-SPI2 (gi.repository.Atspi).
| Approach | Addressing | Survives layout change | Used here |
|---|---|---|---|
| Pixel-only automation (raw xdotool, image matching) | guessed x, y |
❌ brittle | fallback only |
| Accessibility-tree automation | role + name → element | ✅ robust | default |
| Tool | What it does |
|---|---|
screenshot / screenshot_region |
Capture the display (or a sub-rectangle) as PNG; max_dim downscales for faster round-trips |
accessibility_tree |
Actionable elements as {app, role, name, x, y, w, h} in screen coordinates |
click_element |
Find an element by role + name and click its center — no pixel guessing |
enable_accessibility |
Turn on AT-SPI tree export (and silence the screen reader — see safety) |
list_windows |
Windows with id, name, pid, geometry |
focus_window |
Raise a window by id or name; minimizes the frontmost blocker first |
click / move / drag / scroll |
Pointer actions at pixel coordinates |
type_text / press_key |
Keyboard input — press_key uses xdotool syntax (Return, ctrl+c, super+l, KP_0) |
show_cursor |
Visual click cursor overlay: a ring + click ripple showing exactly where the agent is acting |
show_cursor starts a transparent, click-through overlay that draws a cyan ring at the action
point and an expanding ripple on every click, with a label. It exists for human oversight —
you can watch an agent work and interrupt it, instead of guessing what it just did.
Linux with X11, plus:
sudo apt install xdotool ffmpeg gir1.2-atspi-2.0 python3-gi
# at-spi2-core is normally already running on GNOME/KDEffmpeg— screen capture (x11grab)xdotool— mouse and keyboard inputpython3-gi+gir1.2-atspi-2.0— the accessibility tree (import isgi.repository.Atspi, notpyatspi)Pillow(optional) — only formax_dimscreenshot downscaling- GTK 3 (optional) — only for the
show_cursoroverlay
pip install claude-workman # from PyPI
# or from source:
git clone https://github.com/AdvisorAGI/claude-workman.git
cd claude-workman && pip install -e .Run the stdio MCP server:
python -m workman.serverRegister it with an MCP client — for Claude Code:
claude mcp add --scope user workman -- python -m workman.serverTarget a specific X display with WORKMAN_DISPLAY (falls back to DISPLAY, then :0):
WORKMAN_DISPLAY=:1 python -m workman.serverscreenshot— andenable_accessibility+accessibility_treewhen precision matters- Locate — prefer an element by role + name; fall back to pixels
- Act —
click_elementwhen possible, elseclick/type_text/press_key screenshotagain to verify before anything irreversible
Exporting the accessibility tree from Chromium and Electron apps requires the desktop
screen-reader flag — and that same flag makes Orca read the screen aloud through
speech-dispatcher → espeak-ng. enable_accessibility therefore stops Orca immediately after
enabling, using an exact process-name match (pkill -x orca, never pkill -f orca, which
self-matches its own shell command and aborts).
GTK applications export their trees with toolkit-accessibility alone, so the web flag is
opt-in.
An open-source MCP server that lets an AI assistant control a Linux desktop the way a person does — screenshot, locate, click, type, verify — with accessibility-tree targeting so clicks hit real UI elements instead of guessed coordinates.
Browser tools (Playwright, Puppeteer, CDP) drive a web page. claude-workman drives the whole desktop: native apps, terminals, file managers, settings dialogs, and browsers alike.
Yes. Register it with claude mcp add --scope user workman -- python -m workman.server. It
works with any MCP client, not only Claude.
Not yet — X11 today. A Wayland backend and a macOS AXUIElement backend are the natural next steps, and contributions are welcome.
Treat it like handing over mouse and keyboard. Use show_cursor so you can see every action,
prefer a dedicated display or VM for unattended runs, and keep the verify-by-screenshot step
before anything irreversible.
After the Walkman — a small portable thing that just plays. Point it at a display and it works.
No. It is a general desktop-control server, not an anti-bot bypass tool.
Issues and pull requests are welcome — see CONTRIBUTING.md. Good first areas: a Wayland backend, a macOS AXUIElement backend, richer AT-SPI role coverage, and tests.
MIT © 2026 Atmosphere AI — see LICENSE.
Keywords: MCP server, Model Context Protocol, computer use, desktop automation, Linux automation, X11 automation, AT-SPI2 accessibility, GUI agent, AI agent desktop control, xdotool alternative, screenshot automation, Claude MCP connector, agentic computer use.