wait: an agent cannot distinguish "element absent" from "device never gave me a picture"

[thymikee]

agent-device's consumers are AI agents verifying mobile apps. For them the worst failure mode is not a crash — it is a confidently wrong verdict. Today wait makes that easy to produce: a capture that never came back and an element that genuinely is not there are hard to tell apart from the error contract, and the safer of the two is the one that is unlabeled.

This came out of the iOS smoke lane break fixed in #1514. That fix is test-harness-only and deliberately narrow; this issue is the product-side problem it worked around.

What happened, concretely

On a loaded simulator a single snapshot capture exceeded a 1000 ms wait budget. Every attempt returned wait_capture_stalled. The calling loop read that as "element not visible", scrolled three times on no evidence, and failed. The element's actual visibility was never established in either direction — the run reported a location verdict it had no observation to support.

Findings (each verified against the tree at a3ab69a11)

1. Plain absence carries no reason at all.
waitDeadlineExceededError(message, timeoutMs, /* captureTruncated */ false) passes undefined details — src/commands/interaction/runtime/wait-polling.ts:76-92. So the taxonomy is:

outcome	details.reason
first capture stalled	wait_capture_stalled
later capture truncated	wait_deadline_exceeded
element genuinely absent	(none)

The exotic infrastructure cases are labeled; the common, decision-critical verdict is not. An agent branching on details.reason gets undefined exactly when the answer matters most.

2. A capture stall reaches the wire with no retry verdict.
normalizeError hoists details.retriable (packages/kernel/src/errors.ts:159), falling back to retriableForErrorCode(code). waitCaptureStalledError (wait-polling.ts:67-74) sets reason, captureStalled, timeoutMs and hint — but never retriable. RETRIABILITY_BY_CODE (errors.ts:260-264) contains exactly one entry, DEVICE_IN_USE, so COMMAND_FAILED resolves to undefined: not "don't retry", but no verdict at all.

ADR 0010 states the contract as "agents can branch on code, retry on retriable, and follow hint". The most obviously retriable failure in the wait path declines to participate in it. An agent following the documented contract will not retry a transient stall. One-line fix.

3. The classification axis is mechanical, not epistemic.
capture-stalled vs capture-truncated is decided by which capture index hit the deadline (wait-polling.ts:40-56). The question an agent actually needs answered is "did I get at least one readable look at the surface?" — which the code already tracks as sawReadableCapture in createUnreadablePollTracker (wait-polling.ts:94-113) and then uses only for the unreadable-content rethrow.

If the first capture returns in 100 ms and a later one hangs, today's error is capture-truncated with no acknowledgement that the surface was read and the selector was absent at that moment.

4. Absence evidence exists, but is reached by message-string matching.
src/daemon/wait-current-surface.ts appends "Current surface: …" plus details.currentSurface to plain timeouts, and correctly skips it for stalls. Right instinct — an absence verdict should arrive with what was there instead, which is what lets an agent choose scroll vs wrong screen, re-navigate vs report a real bug. But it keys off isWaitTimeoutMessage matching the message string (wait-current-surface.ts:48). That belongs in the taxonomy, not in a regex over prose.

5. wait matches tree presence, not on-screen visibility.
deriveSelectorCapturePolicy (src/commands/interaction/runtime/selector-capture-policy.ts:14-22) includes rects only when the predicate is visible/hidden; a bare wait <selector> has no predicate, so no geometry is captured and waitForSelector never checks any. On iOS the XCUITest tree routinely contains off-screen elements of the current screen, so "wait succeeded ⇒ it is on screen" is accidentally true at best. Callers — including the helper in #1514, whose failure message said "did not become visible" — assume otherwise.

6. The product tells agents to hand-roll the loop that got this wrong.
src/cli/parser/cli-help.ts:264:

Missing target in a long list: use a short manual scroll + snapshot loop with a max attempt count.

Every agent is instructed to reimplement scroll-to-find with its own budget and its own version of this conflation. Our own test helper did exactly that and got it wrong; external agents will do no better.

Proposed direction

Tier 1 — make the contract honest (small, contained)

The organizing invariant:

An absence verdict requires at least one readable capture. A deadline that expires with zero readable captures is always "not observed", never "absent".

sawReadableCapture already exists; it just isn't used for classification. With this in place a mis-set budget degrades into an honest "I could not look" instead of a false location claim, and the exact millisecond value stops being safety-critical — which is the real lesson of the #1514 break, since 2500 ms is itself a guess with the same failure mode at a different threshold.

• waitCaptureStalledError: set retriable: true
• Emit a real reason for plain absence (e.g. wait_target_absent) with readableCaptures and waitedMs
• Classify by readable-capture count rather than capture index; keep not-observed / absent / found-but-wrong distinct
• Fold the wait-current-surface.ts decoration into the taxonomy, retiring the message-string match
• Document the reason enum in the help topics agents are routed to

Tier 2 — own the scroll-search loop (design discussion, not a fix)

A native scroll-to-find capability, so agents and tests stop reimplementing it. scrollUntilTypedMaestroTarget in src/daemon/adapters/maestro/daemon-runtime-port-observation.ts reportedly already does this better than the folklore version — scrolling within the correct scrollable ancestor, settling by snapshot signature between probes, and treating a dead capture as a capture error rather than a "not visible". Worth converging the native side onto it, plus end-of-content termination so absence becomes a spatial proof ("scrolled to the end, N settled captures, no match") instead of an exhausted clock.

⚠️ The Tier 2 reading of the Maestro loop has not been independently verified — it should be confirmed before any work is planned on it. Tier 1 stands on its own and does not depend on it.

Explicitly not proposed

• Adaptive or device-calibrated timeouts — nondeterministic repros and drift, to solve something the Tier 1 invariant solves properly
• Differentiated exit codes — a complete reason enum is the better lever for --json consumers
• Making wait exit 0 on absence — it is used pervasively as an assertion primitive; non-zero with a labeled verdict is the right shape
• A session-cached query layer for cheap absence checks — stale-cache absence is the same wrong-verdict failure from another direction

Origin

Surfaced while fixing the iOS smoke lane (#1514, merged). Design input from a separate model consultation; findings 1-6 above were re-verified directly against the source before filing.

[thymikee]

Triage: confirmed as a product correctness bug and ready for implementation, scoped to Tier 1. The acceptance invariant is: an absence verdict requires at least one readable capture; zero readable captures must remain a retriable not-observed verdict. Please keep Tier 2 (native scroll-to-find) out of the first fix and open a separate design issue only after verifying the Maestro implementation directly.