Skip to content

feat(tools): add screen-fingerprint, a stable answer to "which screen is this" - #727

Closed
hubgan wants to merge 1 commit into
fix/flow-type-requires-focusfrom
feat/screen-fingerprint-tool
Closed

feat(tools): add screen-fingerprint, a stable answer to "which screen is this"#727
hubgan wants to merge 1 commit into
fix/flow-type-requires-focusfrom
feat/screen-fingerprint-tool

Conversation

@hubgan

@hubgan hubgan commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #726.

Why an element is a weak proxy for a screen

Every screen check argent had went through one element of the UI tree, and that is unreliable in four separate ways:

  • a shared header matches on two different screens;
  • a positional id encodes how many siblings exist, not what the thing is;
  • during a push or a modal presentation, source and destination are in the tree together, so a destination landmark matches while the destination is still animating in;
  • the vocabulary of the tree is source-dependent — iOS describe prefers the ax-service and falls back to native-devtools when the AX read comes back empty, flipping labels to testIDs mid-session.

What this adds

screen-fingerprint reads the app's focused React Navigation route path and returns it as "HomeTab>Profile".

Because it comes from the app's own navigation state it is independent of the tree source, of content (/user/alice and /user/bob share a route), and of locale. One route names one screen.

It is read the same black-box way argent already reads the RN component tree: walk the fiber tree via __REACT_DEVTOOLS_GLOBAL_HOOK__ and collect the focused route/navigation prop pairs. No app cooperation is required — the app does not have to export its navigationRef. The probe is self-contained, defensive, and fiber-count bounded so it cannot hang navigation.

A read-only, device-only probe: nothing is tapped, no file is touched.

Two limits, stated rather than papered over

Both are load-bearing for callers, so they are documented in the API and in the skill:

  1. A native overlay above an RN screen — a system permission alert, a share sheet, an RN <Modal> — does not change the focused route, so the probe reports the screen beneath it. Route identity answers "which screen", never "is anything covering it".
  2. Navigation state commits when the navigator commits, which is before the transition finishes animating. It is not a readiness signal either.

Answers that mean different things

  • available: false — this build can never have a reader (release build, fully native app, Chromium, Metro down). Recognize the screen by a destination-only element instead.
  • route: null with available: true — no focused route at this instant: a native screen, or a transition still in flight. Let it settle and probe again.

app_id is required, and guards against reading a foreign app's Metro on the same port.

metroServerRunning

Added alongside, and deliberately not discoverMetro — which also requires at least one CDP target and throws otherwise. That distinction is the whole point: a Metro serving one app reports an empty target list for several seconds after that app relaunches, so "no targets" is the normal post-launch state, not a down server. Answering the up/down question with target discovery told authors to start a server that was already running.

Skill/docs

argent-device-interact gains the tool and its two limits; the routing rules gain a line pointing at it for "which screen is the app on". The await-screen-idle row joins the same table — it documents a tool that already exists and belongs beside this one.

… is this"

Every screen check argent had proxied identity through one element of the UI
tree, and an element is a weak proxy: a shared header matches on two screens, a
positional id encodes how many siblings exist, and during a push or modal
presentation source and destination are in the tree together — so a destination
landmark matches while the destination is still animating in. Worse, the
vocabulary of the tree is source-dependent: iOS `describe` prefers the
ax-service and falls back to native-devtools when the AX read comes back empty,
flipping labels to testIDs mid-session.

The focused React Navigation route path has none of that. It is read from the
app's own navigation state, so it is independent of the tree source, of content
(`/user/alice` and `/user/bob` share a route), and of locale. One route names
one screen.

It is read the black-box way argent already uses for the RN component tree —
walking the fiber tree via the devtools global hook and collecting focused
route/navigation prop pairs. No app cooperation is needed; the app does not have
to export its navigationRef.

Two limits are stated in the API rather than papered over, because callers must
pair around them: a native overlay ABOVE an RN screen (a permission alert, a
share sheet, a Modal) does not change the focused route, and navigation state
commits BEFORE the transition finishes animating. Route identity answers "which
screen", never "is anything covering it" and never "has it settled".

`available: false` is a distinct answer from `route: null`. The first says this
build can never have a reader (release build, fully native app, Chromium, Metro
down) and to recognize the screen by an element instead; the second says there
is no focused route at this instant and to probe again.

`metroServerRunning` is added alongside, and is deliberately not
`discoverMetro`: a Metro serving one app reports an EMPTY target list for
several seconds after that app relaunches, so "no targets" is the normal
post-launch state, not a down server. Answering the up/down question with target
discovery told authors to start a server that was already running.
@hubgan

hubgan commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: the route fingerprint is not being taken forward.

Reading the focused React Navigation route looked like a stable answer to "which screen is this", but it is weaker than it appears. Two screens presented inside one route report the same fingerprint, so a gate on it passes whether or not the navigation happened - and the vacuous case is invisible at authoring time. It also needs a Metro-served debuggable build, so Chromium, release builds and fully native screens fall back to an element check anyway, which is what every flow then has to be written against.

#728 keeps the half that earned its place, await: { idle: true }, and now proves identity with an element that exists only on the destination. That works on every platform and cannot be vacuously satisfied by a shared route.

The metroServerRunning helper added here went with it; nothing else depended on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant