fix(cua-driver): scope the Linux AT-SPI walk to the requested window - #2895
Merged
Conversation
f-trycua
force-pushed
the
fix/atspi-window-scoped-walk
branch
from
August 5, 2026 19:01
290e9d2 to
acdd8d3
Compare
AT-SPI publishes one accessibility tree per process, so a multi-window application exposes every window's controls together. The Linux walker accepted an `xid` and threaded it as far as `walk_tree_bounded_with_timeout`, but `collect_visited_bounded` seeded the traversal from every child of the application and never used it — the xid only reached the geometry pass and the degraded X11-property fallback. Every "window-scoped" Linux snapshot was really application-scoped, and matching a control by label could find one in a window the caller never named. Existing-profile browser setup papered over this by refusing whenever the browser process owned more than one native window, since single-window processes are the case where application scope and window scope coincide. That made the whole route unreachable for ordinary sessions: any second Chrome window — a search window, an OAuth popup, a second profile — was enough to fail it. Resolve the native window to exactly one application top-level instead, by correlating `Component.GetExtents` against the X11 geometry the caller already named. The correlation refuses ties rather than guessing, so callers that act on behalf of an exact window get a proof or a refusal. Setup now demands that proof and drops the cardinality precondition; where window identity genuinely cannot be established (a generic Wayland session with no compositor window list) the same proof fails and setup still refuses, now with a reason that names the cause. Element indices deliberately stay application-wide. They are the contract between a snapshot and every actuator that later consumes one (`perform_action`, `focus_element`, `set_value`, …), and those resolve an index against the whole application; renumbering per window would make a window-scoped snapshot's indices name different elements at actuation time. `render` therefore filters what it emits while still consuming the index. Verified on X11 against a three-window Chrome process (each window returns its own frame, indices 0/1/2 preserved) and a two-window GTK process (2-node dialog and 257-node editor returned separately, and a click by token from a window-scoped snapshot lands on the correct element). Refs #2892 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017zpeGjLTLSALtHsuxKLkNp
f-trycua
force-pushed
the
fix/atspi-window-scoped-walk
branch
from
August 5, 2026 19:02
acdd8d3 to
030c1b9
Compare
…#2899) Existing-profile setup reached its `chrome://inspect` page by synthesizing the URL one keystroke at a time: split the fixed URL at `/#`, send the base through XTEST, press Enter, then hunt for a "Remote debugging" control to cover the fragment. XTEST keysym lookup is keyboard-layout dependent and wlroots virtual-keyboard seats drop punctuation, so `chrome://inspect` could arrive at the omnibox as `inspect` — which Chrome submits as a search query. Nothing downstream distinguished that from a slow-loading setup page, so the run ended in a readiness timeout while leaving the user on a search-results page for "inspect". Adopt the macOS and Windows strategy: transfer the whole URL in one operation instead of per-character. Those adapters write it through the accessibility API (`AXValue`, `ValuePattern::SetValue`), which has no working counterpart here — Chromium's AT-SPI bridge refuses EditableText writes on the omnibox — so the transfer goes through the clipboard, which preserves the property that matters: the exact string arrives at once, with no keysym synthesis to mistranslate. The user's clipboard is saved and restored. The only synthesized keys left are ctrl+t, ctrl+l, ctrl+a, ctrl+v and Enter, all plain letters carrying no layout dependency. Verification is against the destination rather than the input field. Chromium exposes no readable text on its omnibox over AT-SPI — no Value interface, and no Text content even while the field holds a URL — so the read-back those adapters perform cannot be ported. Proving the tab reached the fixed setup page is stronger anyway: it catches a mistyped URL, a hijacked search and a redirect alike, and it reports what was actually reached instead of timing out with an unrelated message. Verified against Chrome 151 on X11 with a real user profile: setup now reaches chrome://inspect/#remote-debugging, matches the exact per-instance checkbox, toggles it, and closes its temporary tab — the whole navigation sequence that previously never got past the omnibox. Not fixed here: after the checkbox is enabled, Chrome 151 exposes no loopback endpoint at all (no listener for the browser pid across 30s, no DevToolsActivePort), so browser_prepare still refuses with browser_requires_setup and rolls the toggle back. That is downstream of navigation and looks like it needs a browser restart to take effect. Refs #2892 Claude-Session: https://claude.ai/code/session_017zpeGjLTLSALtHsuxKLkNp Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 5, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
The Linux AT-SPI walker took an
xidand never used it to scope anything.atspi/mod.rs::walk_tree(pid, xid, …)→native::walk_tree_bounded(pid, xid, …)→walk_tree_bounded_with_timeout(pid, xid, …), whose body calledcollect_visited_bounded(conn, pid, max_elements, max_depth). Thexidreached only the geometry pass and the degraded X11-property fallback. The traversal itself seeded from every child of the application accessible.AT-SPI publishes one tree per process, so this meant every "window-scoped" snapshot on Linux was really application-scoped. For a single-window app the two coincide and nothing looks wrong. For a browser — three windows under one PID is an ordinary Tuesday —
get_window_state(pid, window_id)returned all of them, and matching a control by label could find one in a window the caller never named.Why it surfaced now
Existing-profile browser setup drives Chromium's internal
chrome://inspect/#remote-debuggingpage through accessibility: find the checkbox labelled "Allow remote debugging for this browser instance", toggle it, prove the new loopback endpoint, close the temp tab. With an application-wide tree and several windows open, that label match is not guaranteed to be the window the caller approved — and there can be one such checkbox per open setup page.setup_existing_profile_endpointguarded against that with a cardinality check:Sound, but it made the route unreachable for most real sessions: a second Chrome window of any kind fails it. macOS carries no equivalent precondition (
platform-macos/src/browser/platform.rs) becauseAXUIElementis genuinely window-scoped there — a good sign this was a workaround for the missing primitive rather than a principle.The fix
Resolve the native window to exactly one application top-level, then scope the walk to it.
resolve_window_framecorrelates each frame-role top-level'sComponent.GetExtents(Screen)against the X11 geometry the caller already named. It runs against the sameget_children()list the walk is about to seed from — re-reading it could observe a different window set, and an ordinal resolved against one list but applied to another names the wrong window.correlate_frame_to_windowscores by geometry distance and requires the winner to be within a tolerance and to beat the runner-up by a margin. Decoration offsets resolve; two same-geometry windows refuse rather than coin-flip.Visitednode carries theframe_ordinalit descends from, inherited by children.browser_setup_uinow requires that proof before it matches or actuates any control, and the cardinality precondition is gone. Where window identity genuinely cannot be established — a generic Wayland session with no compositor window list — the same proof fails and setup still refuses, with a message that names the cause and points at--remote-debugging-port.Element indices stay application-wide
This is the load-bearing detail. Indices are the contract between a snapshot and every actuator that later consumes one —
perform_action,focus_element,set_value,scroll_element,type_into_editable_at— and each of those resolves an index against the whole application viacollect_visited(conn, pid). Renumbering per window would make a window-scoped snapshot's indices name different elements at actuation time, silently actuating the wrong control.So
rendergained anonly_framefilter that still consumes the index for skipped nodes and only suppresses emission. A window-scoped snapshot shows one window's nodes carrying their application-wide indices, and actuation keeps resolving them correctly with no changes to any actuator.Verification
Unit — 7 correlation cases (sibling match, decoration offset, ambiguous tie, no match, no candidates, unmapped zero-area extents, sole-candidate-still-has-to-match). Full
platform-linuxlib suite: 227 passed.Live, X11, three-window Chrome under one PID:
52428859"Sign in – Google Accounts"[0] frame "…Your Chrome"52429004"hello – Google Search"[1] frame "…Francesco 2"52429010"Cua: Scale computer fleets…"[2] frame "…Francesco 1"Each window returns only its own frame, with application-wide indices preserved.
browser_preparewithstrategy=existing_profileno longer refuses on cardinality and proceeds to drive the setup UI.Live, two-window GTK process (one Mousepad PID, editor + dialog):
[0] "No",[1] "Yes"[2]Before this change both calls returned all 259 nodes. Then
clickbyelement_tokenminted from the window-scoped dialog snapshot dismissed the dialog — confirming actuation still lands on the right element through the app-wide index space.Not fixed here
Existing-profile setup still cannot complete against this Chrome instance, for a separate and pre-existing reason: Chrome publishes no renderer content to AT-SPI at all (every window walks to a bare frame with zero children), so the setup page's controls never appear. That is the condition
setup_not_ready_messagealready documents — Chromium needs--force-renderer-accessibility, or an AT client that enables full renderer accessibility. Unrelated to window scoping, and it reproduces identically on windows the setup flow never touched.Cross-platform e2e coverage for these browser paths, including non-active tabs and multi-window PIDs, is tracked in #2892.
Refs #2892
🤖 Generated with Claude Code
https://claude.ai/code/session_017zpeGjLTLSALtHsuxKLkNp