Skip to content

v0.3.0 — collection queries + filterable event stream

Choose a tag to compare

View all tags
@javimosch javimosch released this 11 Aug 15:17
· 11 commits to main since this release
v0.3.0
59388bf

Agent-first, CLI-first GUI control for Linux desktops via the AT-SPI2 accessibility tree — the Linux answer to Windows-Use, built in machin (MFL).

It ships zero LLM code. Windows-Use is a Python library with the model inside it; linux-use inverts that — it is a pure sensor/actuator binary and your agent is the loop.

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.3.0/linux-use-v0.3.0-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/

# runtime libraries (present on most GNOME desktops already)
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0

gsettings set org.gnome.desktop.interface toolkit-accessibility true
linux-use doctor

sha256: 1359b0c6ecaaf4173ccf7511feeafa29756c644045fe9d6d38cf9556ab1fea0a

The binary is 154 KB and links only the system AT-SPI/X11/GLib libraries above — no runtime, no interpreter, no dependencies to install.

The loop

linux-use apps                             # what exposes accessibility
linux-use state --app gnome-calculator     # interactive elements, each with a ref
linux-use act 'gnome-calculator:/0/1/0/1/0/0/1/0/22#18124260'
linux-use watch --app gnome-calculator --role editbar --duration 5000

Start with linux-use guide — the whole operator manual is embedded in the binary.

Highlights

  • Acts on widgets, not pixels. act invokes the widget's own accessible action: no focus stealing, no pointer movement, and it works while the window is fully occluded.
  • Stale refs are detected, never guessed. Refs carry a role+name fingerprint; a stale one exits 83 with a suggestion. An element whose identity cannot be proven is reported ref_ok:false rather than emitted as a plausible lie.
  • watch streams real AT-SPI events as NDJSON — filter by app, role or event type, stop after exactly N events or a time limit. Block on a UI change instead of polling.
  • Server-side collection queries when they help. A --role query matches server-side in one round trip (measured 5 ms vs 47 ms on a large app); asking for everything still walks, because that was a wash. method tells you which ran.
  • Incomplete answers are loud. Depth- or budget-truncated reads say so in the JSON and on stderr.
  • Agent-first CLI: JSON on stdout, context on stderr, semantic exit codes (80–89), typed errors with recoverable + suggestions, embedded guide, and a warm-registry daemon. Follows cli-specs.intrane.fr.

Known limitations

  • X11 only for synthetic input. click/key use XTEST, which cannot reach native Wayland clients. The AT-SPI paths (act, type, read, state, watch) still work under Wayland.
  • Accessibility coverage is not universal. GTK/Qt/Chromium/Electron are good; Java needs java-atk-wrapper; Flutter and canvas-drawn UIs may expose nothing. state returning 0 elements means "no accessibility", not "empty window".
  • SHOWING ≠ visible to the user — an occluded widget is still SHOWING.
  • A broad collection query can return elements with ref_ok:false (matched, on-screen, but unpathable). Use --walk when you need a ref for every element.

Full docs and changelog: https://javimosch.github.io/linux-use/

Assets 4
Loading