fix(ios): pin tap-outcome corroboration probes to the baseline's backend

[thymikee]

Why

#1605's corroboration rescues taps that XCTest falsely reports as failed — but its same-backend requirement fails closed exactly where those false negatives happen. The recorded-failure screens are the penalty-boundary screens: the baseline was captured via private-AX under penalty, the corroboration probe comes back via the tree backend, hasMatchingPresentation correctly refuses the cross-backend comparison, and a tap that landed surfaces as XCTEST_RECORDED_FAILURE.

Cost is measured: the AppControlBench bsky-16 success run hit this 4 times in 85 tools, each one sending the model into a re-observe/retry spiral (~16 wasted tool calls). The Bluesky drawer-menu press under penalty is a deterministic repro — it errored in every session tonight while provably landing (ios_tap_failure_corroboration_backend_mismatch: baselineBackend=private-ax, afterBackend=tree).

What

The comparison stays same-backend by design (backends are never comparable views of a screen). Instead, the probe is captured the way its baseline was: a new internal preferredBackend option (never CLI-exposed) threads daemon→runner, and a private-AX-preferred capture takes the exact penalized route — privateAX-first plan, 'deferred' verdict, so no degradation warning renders and settle's budget reset does not fire for a deliberately pre-selected backend. Pinning only engages when the baseline backend is private-ax; tree/queries baselines behave as before.

Evidence

• Live before/after on the deterministic repro (seeded bench feed, drawer press under penalty): before — Error (XCTEST_RECORDED_FAILURE) with the mismatch diagnostic; after — Warning: XCTest reported the tap as failed, but a same-scope post-action accessibility capture changed; treating the tap as landed, and the request diagnostics contain no mismatch phase.
• Daemon tests (13 total in the suite): pinned baseline → the dispatch context carries snapshotPreferredBackend: 'private-ax' and the tap corroborates; tree baseline → no pin; all pre-existing fail-closed cases (cross-backend, sparse, stale, keyless, presentation mismatch) unchanged.
• Swift: the plan gate is the pure snapshotXCTestChannelTreatedAsPenalized, with an in-bundle test asserting pin→penalized-plan composition ([.privateAX], .deferredToIndependentBackend) — added to ios.yml's executed regression list. Ran on-simulator locally: passed.
• Typecheck, layering guard, oxfmt, daemon suites green.

Part of #1599.

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	1.98 MB	1.99 MB	+428 B
JS gzip	634.7 kB	634.8 kB	+105 B
npm tarball	768.3 kB	769.0 kB	+751 B
npm unpacked	2.70 MB	2.70 MB	+2.2 kB

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	27.0 ms	27.2 ms	+0.2 ms
CLI --help	65.0 ms	64.3 ms	-0.7 ms

Top changed chunks:

Chunk	Raw diff	Gzip diff
dist/src/interaction.js	+223 B	+47 B
dist/src/context.js	+97 B	+22 B

[thymikee]

Reviewed exact head c09d5e3a. The regular interactive private-AX path is coherent and the linked live reproduction supports the direction, but this head is not ready:

• P1 — raw private-AX baselines are not actually pinned. Corroboration preserves the baseline presentation, including raw: true, and requests preferredBackend: 'private-ax'. But raw capture uses rawDiagnosticPlan, while effectiveSnapshotCapturePlan honors the penalized/preferred route only when plan == regularVisiblePlan; the raw plan therefore remains tree-first and can return tree, recreating the backend-mismatch false failure. Either explicitly exclude raw baselines from corroboration or make preferred-backend routing pin raw probes without weakening raw diagnostics, and add a raw-baseline regression for the chosen contract.
• P2 — the daemon→runner wire path is not regression-proven. The daemon test stops at a mocked dispatch context; the Swift test calls the pure plan gate directly. Removing handleSnapshotCommand’s forwarding, or the Apple interactor’s preferredBackend RunnerCommand field, leaves both green. Add a provider/transport-level assertion for the emitted snapshot RunnerCommand and a runner-facing test that proves the preferred regular/raw request reaches its applicable plan.

Keep CI classification separate, but do not apply ready-for-human while these code/test gaps remain.

[thymikee]

Reviewed against #1605/#1599 and the capture-reliability contract in CONTEXT.md.

The mechanism is the right one. Backends genuinely aren't comparable views of a screen, so fixing how the probe is captured rather than loosening hasMatchingPresentation is the correct side to change, and pinning only when the baseline is private-ax keeps the blast radius at the penalty boundary where the false negatives actually live. Threading is clean, and I confirmed snapshotPreferredBackend appears in no CLI grammar or option spec — getConfigurableOptionSpecs builds from flag definitions, so project config can't reach it. Worth stating explicitly given GHSA-m7q5-6423-2mwq was exactly about config-derived connection flags.

Four notes, none of them blocking.

1. The other half of the same gate is still fail-closed — and it's on-theme

hasMatchingPresentation rejects on two conditions. This PR fixes the cause of the backend one; the presentation one is still manufactured internally.

matchingCaptureFlags (interaction-ios-tap-outcome.ts:232-236 on this branch) returns undefined whenever params.flags is falsy, discarding the baseline's depth/scope/raw. The probe is then captured at default presentation and presentationKey can't match a non-default baseline — the same "landed tap reported as failure" outcome this PR is closing, through the sibling condition.

The asymmetry is visible in the function you just edited: :109 reads interactiveOnly: presentation?.interactiveOnly ?? true unconditionally, while depth/scope/raw are dropped one line up.

params.flags is req.flags, optional on DaemonRequest, and two production paths omit the key entirely — batch steps (src/cli/batch-steps.ts:89 spreads it away when undefined) and the JSON-RPC boundary (src/daemon/server/http-server.ts:170 passes through whatever the client sent). CLI and Node-client callers always populate it, so this is narrower than the backend case, but it defeats the rescue on exactly the "snapshot --scope <region>, then act inside it" flow.

Fix is small — if (!flags && !presentation) return undefined; plus ...(flags ?? {}). I have it written up with a regression test. Happy to fold it into this PR or stack it behind you; flagging mainly because whichever lands second will conflict in this function.

2. Test 1's behavioural assertions survive a revert

Applying the repo's own rule ("identify what deletion or revert would make it fail"):

In a private-ax baseline pins the corroboration probe to private-ax, the mock returns snapshotPayload(imageViewerNodes, 'private-ax') regardless of the context it's handed. So with the production change reverted, baseline and probe are still both private-ax, hasMatchingPresentation passes, and expect(response?.ok).toBe(true) plus the warning assertion still hold. The only revert-sensitive assertion is snapshotPreferredBackend === 'private-ax' — the wiring, at the layer the change was made.

That's acceptable here, because the Swift test proves pin → plan composition and the live before/after covers the end-to-end binding. But no automated test proves "pin ⇒ the probe actually comes back private-AX-backed", and the mock's hardcoded backend is what hides it. Worth a comment on the test so a later reader doesn't mistake it for that proof.

a tree baseline does not pin the corroboration probe backend passes identically before and after the change — it guards against future over-pinning, which is worth having, but it isn't evidence for this fix.

3. The capture-reliability contract now under-describes the mechanism

CONTEXT.md's corroboration bullet still says the daemon "may take one same-presentation post-action capture against a usable retained snapshot". It doesn't mention that the probe is now captured with a pinned backend on a deliberately penalized plan. That file is the durable vocabulary, and ADR 0004 owns capture-plan/verdict semantics — one line in each would stop the doc drifting from the code, per the repo's own "an ADR conflict is a review finding unless the PR updates it".

4. Nit: hand-mirrored test context builder

contextFromFlags in the test now mirrors daemon/context.ts by hand for a single field. That's a drift surface: if the corroboration path grows a dependence on another context field, the helper silently diverges while the tests stay green. Deriving it from the production builder would be sturdier if that's cheap to do.

---

Not independently verified by me: the claim that settle's budget reset doesn't fire for .deferredToIndependentBackend. I'm taking the live evidence on that one.

The Swift gate itself reads well — snapshotXCTestChannelTreatedAsPenalized is a pure function with a real truth table, preferredBackend: "tree" correctly doesn't pin, and keeping it inside #if os(iOS) matches the isLocalIosRunnerSession guard on the caller.

[thymikee]

Both findings addressed in b2ed38e:

P1 — raw baselines: took the exclusion contract. readCorroborationBaseline declines a raw: true presentation up front (new ios_tap_failure_corroboration_raw_baseline diagnostic) — the raw diagnostic plan keeps tree-first error propagation by design and is never rerouted by penalty or pin, so a raw baseline can never be matched same-backend; bending raw's plan for an evidence probe would weaken exactly the diagnostics raw exists for. Regression: raw private-AX baseline + recorded failure → zero probe captures dispatched, failure surfaces unchanged.

P2 — wire proof at every hop:

• New dispatch-level test drives dispatchCommand with snapshotPreferredBackend in the context and asserts the emitted RunnerCommand carries the pin (and that a bare snapshot doesn't) — removing handleSnapshotCommand's forwarding or the interactor field turns it red.
• The injected-transport suite (interactor-runner-provider.test.ts) asserts the interactor's snapshot payload both ways at the provider seam.
• Runner side: snapshotOptions(from:) extracted as the pure command→options projection, and testDecodedPreferredBackendReachesOptionsAndApplicablePlan decodes the actual wire JSON, projects it, and composes with the plan rule — pinned regular plan → [.privateAX] deferred; RAW plan → untouched; field-less command → no pin. Executed on-simulator (both plan tests passed) and added to ios.yml's regression list.

Layer coverage now: daemon corroboration behavior (mocked dispatch) → dispatch→RunnerCommand (new) → provider transport (new) → decoded command→options→plan (new, executed) — no hop left unpinned.

[thymikee]

Re-reviewed exact head bf2a8d0e. Both prior blockers are fixed. Raw baselines now fail closed before any corroboration probe because the raw diagnostic plan intentionally remains tree-first; the regression proves zero snapshot dispatch. The preferred private-AX backend now has complete production-route evidence across daemon context → snapshot handler → dispatch → Apple interactor/transport → runner JSON decode/options → applicable plan composition, with assertions that would fail if any forwarding hop were removed. Regular private-AX probes are pinned to the privateAX-first deferred plan, while tree/query behavior remains unchanged. The PR correctly scopes itself as Part of #1599, GitHub reports MERGEABLE, completed checks are green, and the body includes deterministic simulator before/after evidence. No actionable findings; code-review clean and eligible for ready-for-human while iOS Smoke completes.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-06 11:27 UTC