Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,5 @@ android/snapshot-helper/build/
android/snapshot-helper/dist/
android/ime-helper/build/
android/ime-helper/dist/
android/multitouch-helper/build/
android/multitouch-helper/dist/
59 changes: 59 additions & 0 deletions docs/adr/0012-interactive-replay.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,12 @@ Accepted (2026-07-10); partially implemented (last updated 2026-07-16). See [Mig
- Decision 6 amendment, repair-segment default exclusion of observation-only commands and the `--record`
opt-in — stage 1 (interim guidance only, #1287) and stage 2 (the semantic default-exclusion fix,
this amendment) of #1271. See the amendment under decision 6 below.
- Decision 3 amendment (#1349), read-only step identity — landmark-mode `target-v1` evidence and
post-resolution (in-loop) verification for selector `wait`, `get`-pattern coverage for `is` (except
`exists`), the `targetIdentityVerification` descriptor trait pinning the evidence-carrying command
set, explicit deferral of read-only `find`/`is exists`/non-selector waits, and ADR 0016's destination
guard strengthened to require a `verified` landmark annotation, with a provider-level
reshuffled-screen false-pass regression. See the amendment under decision 3 below.

**Accepted but NOT yet implemented** (this amendment; tracked by #1235 — repair-transaction lifecycle):
the R7 repair-transaction keep-alive and its distinct `resume.repairSessionHeld` signal, the ARMED →
Expand Down Expand Up @@ -370,6 +376,58 @@ A recorded `id` never matches a node without that id.
> platform-agnostic: an RN `FlatList` row (`Cell`) that is label-less with a labeled `Text` child hits
> the same substitution on iOS.

> **Amendment (#1349): read-only step identity — `wait` landmark verification, `is` coverage, and the
> per-command verification-phase trait.** Decision 3's evidence and machinery extend to the eligible
> read-only steps without changing their execution semantics. The annotation format is **unchanged**
> (`target-v1`, same tuple, same bounds, same writer/parser); what is new is a per-command dispatch of
> WHEN verification runs, declared on the central `CommandDescriptor` as
> `targetIdentityVerification: 'pre-dispatch' | 'post-resolution'` and pinned by a completeness test —
> a new evidence-carrying command must choose its phase explicitly rather than silently entering the
> generic pre-dispatch path.
>
> - **`wait <selector>` — covered, `post-resolution`, landmark (existence) semantics.** A wait's
> landmark may legitimately be absent when its step starts, so the generic pre-action verification
> (paths 2-6) must never run for it; only path 1 (recorded-`unverifiable`, which consults no screen)
> refuses up front. The recorded annotation instead travels into the wait's own polling loop
> (`internal.replayLandmarkGuard`, the same daemon-only channel as `replayTargetGuard`), which
> strengthens the success condition from "the selector matches" to "a selector match carries the
> recorded identity" — the identity tier only (local identity + leaf-anchored ancestry prefix). The
> positional disambiguation signals are deliberately **not** compared: a destination guard proves the
> landmark exists on the ready screen, not that it kept its list position, and requiring position
> would produce the inverse false-failure on a legitimately reshuffled-but-correct screen. Polling is
> preserved fail-open in time and fail-closed at the deadline: a same-selector impostor never aborts
> the wait (a transient look-alike mid-transition is exactly what a wait exists to wait through); at
> the deadline, a poll history containing selector matches that never carried the identity surfaces
> as an `identity-mismatch` divergence (`matchCount` from the last matching poll, observed identity +
> first ancestry mismatch in `mismatches`) BEFORE the wait reports success, while a selector that
> never matched at all remains the ordinary wait timeout (`action-failure` — that failure needs a
> state repair, not an identity repair, and its `repairHint` routing is already correct).
> A wait poll also rides out a capture that judged the current screen UNREADABLE (the Android
> helper's content verdicts, `isUnreadableCaptureContentError`; iOS surfaces the same state as a
> sparse verdict with no matches): live Android validation showed a guard wait replayed right after
> a navigation press deterministically dies on its first mid-transition capture otherwise. A wait
> whose screen never became readable rethrows the last capture verdict at the deadline instead of
> masking it as a generic timeout.
> Record time writes evidence in a **landmark mode**: the step-5 self-check is identity-set
> MEMBERSHIP rather than isolation (mirroring what replay will actually verify), and a matched node
> with no id and no label after #1269 demotion records **no annotation** — a role-only landmark
> identity is near-vacuous, and an unannotated wait keeps its existing selector-existence semantics
> instead of failing closed on evidence that never discriminated anything. ADR 0016's destination
> guard consumes exactly this: a qualifying guard is a selector wait with a `verified` annotation.
> - **`get` — unchanged**; already covered by the pre-dispatch path and the post-resolution guard.
> - **`is` (all predicates except `exists`) — covered, `pre-dispatch`, the `get` pattern end-to-end.**
> `is` resolves a unique node immediately, so pre-action verification is semantically valid; the
> resolved node/tree feed record-time evidence, and dispatch threads `replayTargetGuard` into
> `assertExpectedResolvedTarget` exactly like `get`. The direct-iOS `is`/`wait` fast paths are gated
> off during recording and guarded replays, mirroring `get`'s existing recording gate.
> - **Intentionally deferred, with tests proving no annotation is recorded and no identity check runs:**
> `is exists` (existence assertion with no unique winner; wait-like semantics without the
> guard-critical role), every read-only `find` variant (fuzzy-locator resolution has no
> selector-chain identity token for the classifier, and publication already refuses mutating `find`
> as non-verifiable), and `wait text`/`wait stable`/duration waits/`wait @ref` (no element target, or
> a session-local ref that ADR 0016 already refuses to publish; `wait @ref` is rejected rather than
> converted to a portable selector).

**Ancestry.** The chain is the nearest **K = 8** ancestors of the target, ordered **leaf→root** (nearest
ancestor first), each entry `{ role, label? }` under the same normalization (`role` may be the empty
string when the node has no type; `label` is omitted when empty). Truncation drops entries from the
Expand Down Expand Up @@ -1254,6 +1312,7 @@ not restate or amend the decisions.
| 8. Agent-supervised re-record repair, base | 6 (R1-R6) | Shipped | #1228 |
| 9. Repair-transaction lifecycle | 6 (R7) | Shipped | #1235 |
| 10. Repair-segment default exclusion + `--record` | 6 amendment | Shipped | #1271 stage 2 |
| 11. Read-only step identity (wait landmark verification, `is` coverage) | 3 amendment (#1349) | Shipped | — |

Step 4 (#1209) added the `selector-miss`/`identity-mismatch`/`identity-unverifiable` divergence kinds
and a post-resolution target guard that cross-checks the dispatched winner against the verified member.
Expand Down
32 changes: 23 additions & 9 deletions docs/adr/0016-active-session-script-publication.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,15 +87,27 @@ the serialized guard before filesystem work and never relies on repair-only bare
`session save-script` refuses publication without this **destination guard** and tells the author to
record one. V1 does not infer a screen identity from a snapshot or synthesize an implicit guard.

The initial guard is selector-level. `wait` does not yet carry recorded-landmark identity through its
polling resolution, so the guard proves that an element matching its selector exists, not that it is the
same landmark element observed while authoring. Authors must choose a selective destination-specific
landmark; a reshuffled screen containing the same weak label elsewhere can false-pass.
[#1349](https://github.com/callstack/agent-device/issues/1349) owns the identity design for waits and
remaining read-only steps. It must preserve polling: ADR 0012's current pre-action `target-v1`
> **Amendment (#1349, shipped).** The guard is now identity-level, not merely selector-level. A
> selector wait recorded while ARMED captures landmark-mode `target-v1` evidence for the matched
> element (ADR 0012's read-only-step amendment), and a qualifying destination guard is exactly a
> selector wait whose annotation is `verification: "verified"` — a selector wait on an identity-empty
> element (no id and no label) records no annotation and does not qualify, so publication refuses it
> with a recovery hint naming a labeled/id-bearing landmark. On replay, polling is preserved: the wait
> keeps polling until a selector match carries the recorded identity (local identity + leaf-anchored
> ancestry prefix), and a deadline reached with only same-selector impostors fails closed as an
> `identity-mismatch` `REPLAY_DIVERGENCE` before the wait reports success. The reshuffled-screen
> false-pass below is covered by a provider-scenario regression (record → publish → replay against a
> reshuffled tree whose same-label node sits under a different id/ancestry).

The original selector-level caveat, retained as context: the guard proved that an element matching its
selector exists, not that it is the same landmark element observed while authoring, so a reshuffled
screen containing the same weak label elsewhere could false-pass.
[#1349](https://github.com/callstack/agent-device/issues/1349) owned the identity design for waits and
remaining read-only steps. It preserves polling: ADR 0012's pre-action `target-v1`
verification cannot be attached to a wait unchanged because a not-yet-present landmark is the expected
starting condition. Any identity check for a wait happens after its selector resolves, before the wait
reports success, or uses a distinct guard-specific mechanism.
starting condition, so the identity check runs after the wait's selector resolves and before the wait
reports success (see ADR 0012's #1349 amendment for the mechanism and the read-only-step coverage
classification).

The phrase "last mutating action" is derived from a request-sensitive recording-effect trait on the
central `CommandDescriptor`, required for every command that records session actions and guarded by a
Expand Down Expand Up @@ -203,7 +215,9 @@ executing that script, not the artifact being saved.
- The artifact replays from a cold start, completes its destination guard, returns the live session id,
and accepts a subsequent command on that session.
- Every action in ADR 0012's existing target-binding command set has canonical identity evidence and no
unresolved `@ref` reaches disk; the destination guard remains selector-level until #1349 lands.
unresolved `@ref` reaches disk; since #1349 the destination guard additionally requires verified
recorded landmark identity, with a reshuffled-screen false-pass regression proving replay fails
closed.
- Existing-target refusal preserves the original bytes; `--force` replaces atomically; a failed publish
remains retryable.
- After PUBLISHED, later ordinary actions remain usable, repeated `session save-script` fails, plain
Expand Down
2 changes: 1 addition & 1 deletion src/cli/parser/cli-help.ts
Original file line number Diff line number Diff line change
Expand Up @@ -234,7 +234,7 @@ Reusable open-to-destination scripts:
agent-device press 'id="continue"' --settle
agent-device wait 'role="heading" label="Screen X"'
agent-device session save-script
session save-script [path] [--force] publishes the sole recorded open through the destination guard, omits close, and leaves the session active. A duration wait, wait stable, or wait @ref is not a destination guard. A second successful open aborts publication; start a fresh session to author again.
session save-script [path] [--force] publishes the sole recorded open through the destination guard, omits close, and leaves the session active. The guard is a selector wait on a labeled or id-bearing landmark: its recorded identity is captured while armed, and replay verifies that identity after the wait's selector resolves, so a reshuffled screen with the same label elsewhere fails closed instead of false-passing. A duration wait, wait stable, wait @ref, or a selector wait on an unlabeled element is not a destination guard. A second successful open aborts publication; start a fresh session to author again.
Recorded fill/type inputs are written literally to the .ad file. Do not record passwords, tokens, or other secrets; use pre-authenticated test state or non-secret fixture credentials until parameterized input authoring is available.

Snapshots and refs:
Expand Down
Loading
Loading