v0.7.1 — the agent skill ships with the tool

The agent-facing skill now ships with the tool, and writing it exposed a bug the skill would otherwise have documented falsely.

An agent skill in the repo

.agents/skills/linux-use/SKILL.md — a drop-in skill for Claude Code, Devin, or anything that reads SKILL.md. Versioned alongside the commands it documents, so it cannot drift.

It leads with when not to use this tool: terminal TUIs belong to tmux, session-free scraping to Playwright, and sending a message to an API. GUI automation is the last resort, and the skill says so before it says anything else.

Also included: the loop, both ref kinds, seven landmines that will actually bite, the verified Teams recipe, and an explicit statement of what is untested.

contrib/a11y-browser

A launcher for a Chromium-family browser that linux-use can actually drive — --force-renderer-accessibility plus its own profile, since the flag cannot be applied to a running browser and PWA windows share a process with ordinary ones.

contrib/a11y-browser -n teams https://teams.microsoft.com/
contrib/a11y-browser -n teams --hidden <url>   # private Xvfb + private XDG_RUNTIME_DIR
contrib/a11y-browser -n teams --stop

The bug it surfaced

find --app <name> returned refs like Microsoft Edge:/0/1#fp even when two processes published that name — so every ref it handed back failed to resolve with app_ambiguous. state refused ambiguity correctly; find quietly did not.

• find now qualifies emitted refs as <name>#pid<PID>:... when a name is shared, and leaves them bare when it is not.
• splitref splits the fingerprint on the last #, so an app part can carry #pid<N>.
• collect() compares against the raw app name, or every ref silently downgraded from a path ref to a geometry ref.

Verified: an ambiguous name now yields path-shaped, pid-qualified refs that read and act correctly; unambiguous apps still emit bare refs; state still refuses a plain ambiguous --app with exit 80.

One more thing that would have shipped broken

.gitignore contained a bare linux-use (for the built binary). Git matches that against any path component, so .agents/skills/linux-use/ was silently excluded — the skill would have been missing from the repo without a single warning. Now anchored to /linux-use.

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.7.1/linux-use-v0.7.1-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0 xclip
gsettings set org.gnome.desktop.interface toolkit-accessibility true
linux-use doctor && linux-use guide

sha256: 6f8fef73a719734e70dc1d4e227ffc24c158dd660dcea8c8824e74e482a69274

Full Changelog: v0.7.0...v0.7.1

v0.7.0 — an honest capability statement

A release about saying what this tool can and cannot do, and what was actually tested. No behaviour changes since v0.6.3 — the code was fine, the documentation overclaimed.

What changed

The README listed "GTK/Qt/Chromium/Electron" as good coverage. Only GTK and Chromium had ever been exercised. Qt has now actually been tested; Electron and Java have not, and now say so.

Target	Status	Evidence
GTK 3/4 apps	verified	gnome-calculator, gnome-text-editor, gnome-control-center, gnome-disks, seahorse, Nautilus — read, act, type, watch
Qt 5 apps	verified	QDBusViewer: 12 interactive elements; act fired ShowMenu and the File menu opened
Chromium/Edge web content	verified, needs --force-renderer-accessibility	test page exposed 41 elements incl. real DOM; Microsoft Teams navigated, composed, sent
Chromium/Edge without the flag	broken by design	exposes exactly one frame — zero web content
Electron	not tested	expected to need the same flag
Java (Swing/AWT)	not tested	needs java-atk-wrapper
Flutter / canvas / games	expected to expose nothing	not tested
Wayland	click/key do not work	XTEST cannot reach native Wayland clients; AT-SPI paths still work (#1)

The honest caveats

• Tested on exactly one configuration: Ubuntu 22.04.3, GNOME 42.9, X11, at-spi2-core 2.44.0, x86-64. Wayland, KDE/XFCE, other distros, HiDPI/fractional scaling, multi-monitor, non-US keyboard layouts and arm64 are all untested.
• There is no automated test suite. Every number in the README is a real measurement, but nothing re-checks them on commit. test/env.sh is an isolation harness, not a test runner. This is now the top item under Next, ahead of Wayland.
• It can break the apps it drives. While building it, this tool crashed a browser three times before the cause was isolated, and once knocked out the developer's own session accessibility bus (a test-isolation gap, since fixed). Both are documented rather than quietly patched.
• Two previously published measurements were wrong and were corrected in later notes — a "zero memory growth" claim read off too short a run, and a "regression" that turned out to predate the change blamed for it.

What it is good at

Driving GTK and Qt desktop apps, and web apps in a suitably-launched Chromium, by name rather than by pixel — with refs that go stale loudly (exit 83) instead of silently acting on the wrong widget, truncated reads that announce themselves, and an event stream you can block on instead of polling. It ships zero LLM code: your agent is the loop.

If you are on Wayland, or need Electron/Java, this is not ready for you yet.

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.7.0/linux-use-v0.7.0-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0 xclip
gsettings set org.gnome.desktop.interface toolkit-accessibility true
linux-use doctor && linux-use guide

sha256: caaa7ac615f29b9ba8e8f5027ffa53c68302f1d0123973358a227dd06b103720

Start with linux-use guide — the whole operator manual is embedded in the binary.

Issues and PRs welcome, particularly from anyone on Wayland, KDE, or a non-GNOME desktop: those are the gaps.

Full Changelog: v0.6.3...v0.7.0

v0.6.3 — keysym case fix + app disambiguation

Two correctness fixes, both cases of the tool giving a confident wrong answer instead of refusing.

Ambiguous --app is refused, not guessed (#3)

Two applications can publish the same accessible name — two browser profiles both call themselves Microsoft Edge. --app silently resolved to whichever the desktop listed first.

$ linux-use apps
  {"name":"Microsoft Edge","pid":872010,"toolkit":"Chromium","windows":1,"ref":"Microsoft Edge#pid872010"}
  {"name":"Microsoft Edge","pid":924858,...,"ref":"Microsoft Edge#pid924858"}
  "ambiguous_names":["Microsoft Edge"]

$ linux-use state --app 'Microsoft Edge'               # app_ambiguous, exit 80
$ linux-use state --app 'Microsoft Edge#pid924858'     # 39 elements
$ linux-use state --app 'Microsoft Edge' --pid 924858  # same

apps now reports pid, toolkit and a ready-made selector, flags ambiguous_names and warns on stderr. windows and find honour the same filter. An absent app still reports app_not_found (82), distinct from ambiguous (80).

Verified with two same-named Edge instances: 375 vs 39 elements, and a heading unique to the second instance is found only under its own pid.

Case-sensitive keysyms, and --dry-run (#2)

key lowercased the whole combo before lookup, but X keysym names are case-sensitive — so alt+F4 failed with unknown_key, while aliases like enter masked the problem for the keys people test first. Closing a window, the obvious move for an app exposing no accessible action, was impossible.

Resolution now tries the alias table (case-insensitive) → the token exactly as written → the token capitalised.

combo	resolves to
alt+F4 / alt+f4	Alt_L + F4
ctrl+F12, Print, Menu, shift+Insert, Pause	as written
ctrl+shift+t, enter, esc, pageup, super+left	unchanged

Also added f1–f24 in either case plus insert, print, menu, pause, capslock, numlock, scrolllock, altgr aliases, and:

linux-use key alt+F4 --dry-run   # {"keys":["Alt_L","F4"],"sent":false}

so a combo can be validated without closing a window to find out.

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.6.3/linux-use-v0.6.3-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0 xclip
linux-use doctor && linux-use guide

sha256: e58b6270f310e38cb2814e8330b1b693a09d9559b0c8fc018539e4a4d7c0afd5

Full Changelog: v0.6.2...v0.6.3

v0.6.2 — clipboard text entry (fixes the Chromium crash)

Fixes #4: the default text-entry path crashed the browser it was typing into.

The bug

AT-SPI key synthesis (atspi_generate_keyboard_event) crashes Chromium/Edge launched with --force-renderer-accessibility. Reproduced on a bare <input> page with focus verified — 3/3 times, payloads of 11/16/112 chars. Since that flag is the only way to make a browser expose web content to AT-SPI, the crash landed exactly where the tool is most useful. sendtext even reported ok:true while the target died.

The fix

Text entry now goes through the clipboard by default: set the CLIPBOARD selection, send ctrl+v via XTEST, restore the previous clipboard. That path never touches the AT-SPI input controller.

Measured A/B on the repro page, focus confirmed on the input:

mechanism	result
default (clipboard+ctrl_v)	text lands, browser alive
sendtext --via atspi	browser dead

• sendtext --via auto|clipboard|atspi — auto (default) prefers the clipboard when xclip is present, and warns loudly on stderr when it must fall back.
• type <ref> uses the clipboard for its fallback, and now verifies the element actually took focus first — grab_focus silently does nothing on some web elements, and typing into an unknown focus is how a stray space presses whatever button is focused.
• paste <text> remains the explicit clipboard command.
• doctor reports clipboard_backend and text_entry_actual, and flags a missing xclip as a problem rather than letting it silently degrade.

Upstream

Not linux-use's bug to fix — worth reporting to at-spi2-core/Chromium once a minimal C repro exists. Environment: Ubuntu 22.04.3, at-spi2-core 2.44.0, Edge 151.0.4129.72, X11.

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.6.2/linux-use-v0.6.2-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0 xclip
linux-use doctor && linux-use guide

sha256: 60e874cfdad46218e794a1306b1564be68ad97386b0eac957c2831263a780a07

Full Changelog: v0.6.0...v0.6.2

v0.6.0 — geometry refs, no more walk fallback

The collection+walk fallback is gone. Every collection match is now addressable, so reads never need a tree walk to complete an answer.

What the fallback actually was

v0.5 fell back to a full walk whenever a match could not be pathed — 14 of 37 elements in gnome-control-center. Diagnosing it found something more interesting than an identity bug: broken parent/child reciprocity.

NODE   filler '' [472,112 578x100]
parent panel  '' [472,112 578x100]  children=1  index_in_parent=-1
  -> pointer match: -1 | role+name+extents match: -1

The node's get_parent() returns a container that does not list it among its children — by pointer, by index, or by role+name+extents. A child-index path to such a node is impossible, not merely unknown. No better identity test could have fixed it (and v0.3 already tried one, atspi_accessible_get_id, which produced wrong refs).

Geometry refs

Those nodes now get a second ref kind:

app:@<roleId>~<x>,<y>,<w>,<h>#<fingerprint>

Resolved with a collection query for that role, matched on exact extents, and verified by the same role+name fingerprint. A rectangle with no match returns exit 84; two widgets sharing a role and rectangle are refused as ambiguous rather than guessed. ref_kind on every element says path or geometry.

Verified: all 14 geometry refs in gnome-control-center resolve to the widget they describe, and act through one drives the app correctly.

Caveat: a geometry ref is tied to screen position, so moving or resizing the window invalidates it — exit 84, never a wrong hit. Re-read state after a move.

Results

app	method	default	--walk	geometry refs
gnome-calculator	collection	15 ms	65 ms	0
gnome-text-editor	collection	12 ms	53 ms	0
gnome-control-center	collection	25 ms	44 ms	14

All three return the same element set as the walk, with every ref usable. gnome-control-center was 76 ms via the old fallback.

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.6.0/linux-use-v0.6.0-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0
gsettings set org.gnome.desktop.interface toolkit-accessibility true
linux-use doctor && linux-use guide

sha256: 9c5ec79dd9ecd820eb504b86f0818d315d81b54c806058def7339442be2c66e7

Docs: https://javimosch.github.io/linux-use/

Full Changelog: v0.5.0...v0.6.0

v0.5.0 — bounded watch memory (and where the leak actually lives)

v0.5 set out to close watch's residual memory growth. It instead proved the growth is not linux-use's — and made it bounded.

The finding

A probe with three callback bodies, every allocation inside an arena block including its own reporting:

callback	kB/event
null (only increments a counter)	+0.163
props (reads role + name)	+0.157
full (+ parent climb)	+0.181

A callback that does nothing leaks at the same rate as the full one. The growth is inside libatspi's event delivery on at-spi2-core 2.44, not in this tool. The historical AtspiEventListener leak was fixed upstream in 2.21.1; this is a separate residual.

atspi_accessible_clear_cache() was tried as a mitigation and makes it worse: +0.886 kB/event and roughly half the event throughput.

What v0.5 ships instead

watch --max-rss <kB> — a memory ceiling. Above it, watch exits cleanly with code 90 (rss_limit) after a stderr notice, so a supervisor can restart it and keep streaming:

while :; do
  linux-use watch --app firefox --max-rss 262144 || [ $? -eq 90 ] || break
done

Off by default. At 0.14 kB/event a 256 MB ceiling is roughly 1.8 million events, so most agent workloads never reach it — bound short runs with --duration/--max-events and this never comes up.

Also

• New exit code 90 rss_limit, documented in guide, help-json and the README.
• lu_rss_kb() C helper, because machin's read_file returns empty for procfs.
• No change to the query path; a fresh gnome-calculator still reads in 16 ms (--walk: 72 ms).

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.5.0/linux-use-v0.5.0-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0
gsettings set org.gnome.desktop.interface toolkit-accessibility true
linux-use doctor && linux-use guide

sha256: 22b57f60db89cdd052021e9c9d0f97a320bec6b19b7a99f12529a42013668105

Docs: https://javimosch.github.io/linux-use/

Full Changelog: v0.4.0...v0.5.0

v0.4.0 — memoized path reconstruction

Collection matches are siblings that share ancestors, so reconstructing each match's reference re-walked the same chain once per match — which ate the entire benefit of the server-side query. v0.4 memoizes that climb, and a collection query becomes the default for every read.

Measured across six applications

app	default	method	--walk
gnome-text-editor	12 ms	collection	71 ms
gnome-calculator	23 ms	collection	97 ms
gnome-disks	5 ms	collection	15 ms
seahorse	15 ms	collection	29 ms
Nautilus	560 ms	collection	625 ms
gnome-control-center	76 ms	collection+walk	69 ms
gnome-control-center --role "push button"	4 ms	collection	47 ms

Five of six are 1.1–5.9x faster with identical refs and zero wrong refs. The sixth cannot prove a path for every match, so the read is automatically re-run as a walk (method: "collection+walk") at ~10% more than walking directly — the price of never returning an incomplete set by default.

--collection keeps the fast result including ref_ok:false entries; --walk skips the query. method always reports what ran.

Also in this release

• The path cache is per-query and never survives a UI change — a cached path outliving the tree it described would resurrect exactly the wrong-ref bug this tool exists to prevent.
• watch runs uncached: one climb per event gains nothing from a cache, and its maps would live outside the arena.

Correction to the v0.2/v0.3 notes

The README previously claimed the arena fix reduced watch memory growth to zero. That was wrong — it was read off two samples that happened to coincide over a short run. Measured across ~1000 events, the arena fix cut growth from ~17 kB/event to ~0.17 kB/event (~100x), but not to zero. v0.3 and v0.4 grow at the same rate (~190 kB per 1000 events). Bound long-lived listeners with --duration/--max-events; closing the remainder is the top item for v0.5.

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.4.0/linux-use-v0.4.0-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0
gsettings set org.gnome.desktop.interface toolkit-accessibility true
linux-use doctor && linux-use guide

sha256: c54cbf11acc1cf3f935ad9fddae3eb6861ad4f35e956a442f1425599bb1c603f

Docs and changelog: https://javimosch.github.io/linux-use/

Full Changelog: v0.3.0...v0.4.0

v0.3.0 — collection queries + filterable event stream

Agent-first, CLI-first GUI control for Linux desktops via the AT-SPI2 accessibility tree — the Linux answer to Windows-Use, built in machin (MFL).

It ships zero LLM code. Windows-Use is a Python library with the model inside it; linux-use inverts that — it is a pure sensor/actuator binary and your agent is the loop.

Install

curl -L -o linux-use https://github.com/javimosch/linux-use/releases/download/v0.3.0/linux-use-v0.3.0-x86_64-linux
chmod +x linux-use && sudo mv linux-use /usr/local/bin/

# runtime libraries (present on most GNOME desktops already)
sudo apt install libatspi2.0-0 libxtst6 libx11-6 libglib2.0-0

gsettings set org.gnome.desktop.interface toolkit-accessibility true
linux-use doctor

sha256: 1359b0c6ecaaf4173ccf7511feeafa29756c644045fe9d6d38cf9556ab1fea0a

The binary is 154 KB and links only the system AT-SPI/X11/GLib libraries above — no runtime, no interpreter, no dependencies to install.

The loop

linux-use apps                             # what exposes accessibility
linux-use state --app gnome-calculator     # interactive elements, each with a ref
linux-use act 'gnome-calculator:/0/1/0/1/0/0/1/0/22#18124260'
linux-use watch --app gnome-calculator --role editbar --duration 5000

Start with linux-use guide — the whole operator manual is embedded in the binary.

Highlights

• Acts on widgets, not pixels. act invokes the widget's own accessible action: no focus stealing, no pointer movement, and it works while the window is fully occluded.
• Stale refs are detected, never guessed. Refs carry a role+name fingerprint; a stale one exits 83 with a suggestion. An element whose identity cannot be proven is reported ref_ok:false rather than emitted as a plausible lie.
• watch streams real AT-SPI events as NDJSON — filter by app, role or event type, stop after exactly N events or a time limit. Block on a UI change instead of polling.
• Server-side collection queries when they help. A --role query matches server-side in one round trip (measured 5 ms vs 47 ms on a large app); asking for everything still walks, because that was a wash. method tells you which ran.
• Incomplete answers are loud. Depth- or budget-truncated reads say so in the JSON and on stderr.
• Agent-first CLI: JSON on stdout, context on stderr, semantic exit codes (80–89), typed errors with recoverable + suggestions, embedded guide, and a warm-registry daemon. Follows cli-specs.intrane.fr.

Known limitations

• X11 only for synthetic input. click/key use XTEST, which cannot reach native Wayland clients. The AT-SPI paths (act, type, read, state, watch) still work under Wayland.
• Accessibility coverage is not universal. GTK/Qt/Chromium/Electron are good; Java needs java-atk-wrapper; Flutter and canvas-drawn UIs may expose nothing. state returning 0 elements means "no accessibility", not "empty window".
• SHOWING ≠ visible to the user — an occluded widget is still SHOWING.
• A broad collection query can return elements with ref_ok:false (matched, on-screen, but unpathable). Use --walk when you need a ref for every element.

Full docs and changelog: https://javimosch.github.io/linux-use/