You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reframes iOS snapshot recovery around the XCTest accessibility channel instead of a simulator-only tree backend penalty. Regular snapshots now avoid all XCTest-backed tiers after slow or timed-out XCTest accessibility work and derive recovery from non-XCTest backend traits, while raw diagnostics keep tree-first behavior.
Improves back --in-app by trying native navigation buttons, then semantic top-band Back/Close/Cancel controls, before falling back to a bounded top-leading tap. This avoids false-positive coordinate success on React Native custom search headers.
Validation
pnpm build
pnpm build:xcuitest
git diff --check
Verified on thymikee-iphone with Bluesky: snapshot Explore, focus search by ref, type agent-device, run back --in-app, and confirm the follow-up snapshot returned to Explore.
Reviewed the full diff plus the surrounding runner code it touches. The core direction is sound and one part is genuinely elegant, but there are two behavioral/design issues in the two halves of this PR that I'd want resolved before merge, plus a couple of cleanups.
What's good (keep it)
The SnapshotBackendKind.usesXCTestAccessibilityChannel trait is the right move. It collapses scattered kind == .recursiveTree / kind != .privateAX checks into one semantic property, so effectiveSnapshotCapturePlan, shouldSkipSnapshotBackendForAbandonedTreeCapture, and the record helpers now all read off the same concept. The rename (tree backend → XCTest channel) is accurate, and deriving the recovery plan via plan.filter { !$0.usesXCTestAccessibilityChannel } instead of a hardcoded [.privateAX, .querySweep, .recursiveTree] is the correct generalization. The !recoveryPlan.isEmpty guard is a good defensive touch. The pure static helpers on the back side are well unit-tested.
1. (Blocking-ish, behavior) On a physical device the derived recovery plan is always [.privateAX] — and privateAX is a compile-time no-op there. So the widened penalty turns every snapshot sparse for 120s, including on healthy screens.
This is the interaction of two changes in the PR:
The penalty gate widened from #if os(iOS) && targetEnvironment(simulator) → #if os(iOS) (physical devices now reorder).
privateAX is the only non-XCTest backend, and privateAXSnapshotCapture is #if os(iOS) && targetEnvironment(simulator) — on device its body compiles to return nil.
So on device, effectiveSnapshotCapturePlan under penalty always yields exactly [.privateAX], which always returns nil, which falls straight through to the sparse fallback payload. Concrete sequence:
First hostile snapshot grinds ~30s (unchanged), trips penalizeSnapshotXCTestChannel (120s window).
Every subsequent snapshot for that bundle for 120s → plan [.privateAX] → nil → sparse root, instantly.
Navigate to a healthy screen within that window → still [.privateAX] → still sparse, even though recursiveTree would have returned a real tree. Nothing clears the penalty early, and because the only recovery tier no-ops we never re-probe to discover the screen recovered.
On simulator this is fine (privateAX returns real nodes). On device it means "one hostile screen → 2 minutes of empty snapshots on every screen of that app." That's a real regression vs. main, where device was never penalized and would have captured the healthy screen. It also makes the state: "recovered" / AGENT_DEVICE_RUNNER_SNAPSHOT_RECOVERED stamping misleading on device — nothing is recovered, we just gave up fast.
The trait-filter is elegant precisely because it hides which backends exist per platform — but here it silently produces a plan of exclusively non-functional backends. If fail-fast-sparse is the intended device tradeoff (the ADR edit hints it is), please make it explicit rather than emergent from a compile-time no-op — e.g. detect "recovery plan has no runnable backend on this platform" and either (a) use a much shorter device penalty, or (b) re-probe with a small XCTest budget instead of blanket-sparse for the full 120s. At minimum, a device-path unit test pinning this behavior would keep the invariant honest.
2. (Blocking-ish, design contradiction + latency) back --in-app tier 2 leans right into the XCTest channel the rest of the PR treats as hostile — with no budget.
topNavigationBackElement does app.buttons.matching(predicate).allElementsBoundByIndex and then per-candidate .exists / .isHittable / .frame / .label / .identifier — every one a synchronous XCTest round-trip, ~5 per button, no deadline. That's the same enumeration channel the issue reports grinding to 30s on these RN screens. On main, back only touched navigationBars.buttons (empty → fast 399ms fail). This PR can turn that fast fail into a multi-second grind on exactly the screens #1156 is about.
There's a nice code-judo move that also unifies the two halves' model of the channel: consult the penalty you just generalized. If isSnapshotXCTestChannelPenalized(bundleId:) is set, skip tier 2 entirely and go straight to the coordinate fallback (and/or wrap the enumeration in a small runMainThreadWork budget). One half of the PR learns the XCTest AX channel is dangerous; the other half should get to use that knowledge instead of re-discovering it the slow way.
3. (Design question) Tier 3 makes back --in-app report ok: true for a blind top-leading tap that may have hit nothing — the honest "not available" signal is gone.
tapTopLeadingNavigationFallback returns true whenever the coordinate tap performs (which it essentially always does). So on iOS tapInAppBackControl now almost never returns false, and the caller almost never emits in-app back control is not available. The PR summary says the goal is to avoid false-positive coordinate success on RN custom search headers — but tier 3 just relocates that false-positive to the top-left corner (which on a search header may be the field, a logo, or empty). If "always attempt something and let the follow-up snapshot verify" is intended, fine — but it's worth a comment saying so, because as written it trades a fast honest failure for an unconditional optimistic success, which is the opposite of what the summary claims.
Cleanups (not blocking)
4. Two sources of truth for the back/close/cancel keyword set. The 6-clause predicate (label CONTAINS[c] %@ OR identifier CONTAINS[c] %@ … ×3) and navigationBackControlRank independently hardcode the same three keywords in the same priority order. Add "dismiss" to one and forget the other and they silently drift. Collapse to a single ordered table and drive both from it, e.g.:
staticletnavigationBackKeywords=["back","close","cancel"] // index = rank
staticfunc navigationBackControlRank(label:String, identifier:String)->Int?{lettext="\(label)\(identifier)".lowercased()return navigationBackKeywords.firstIndex{ text.contains($0)}}
and build the predicate by flat-mapping the same list into label/identifier CONTAINS[c] clauses. Removes the duplication and the wall of repeated %@.
5. File size / decomposition. This lands the navigation-back cluster in RunnerTests+Interaction.swift, pushing it to 1625 lines. The cluster (topNavigationBackElement, navigationBackControlRank, isTopNavigationControlFrame, topLeadingNavigationFallbackPoint, tapTopLeadingNavigationFallback + 4 tests) is cohesive and mostly pure — a clean RunnerTests+Navigation.swift. This also lines up with the in-flight issue-1110-interaction-test-split work; worth extracting rather than growing the grab-bag further.
6. Uncommented magic constants. The header-band and fallback-point clamps (0.22/96/180, 0.08/28/44, 0.155/56/132) have zero rationale, in contrast to the heavily-commented snapshot side. Worth a one-liner each. Note the y term is effectively dead on standard phones: for a 932pt-tall frame, 932*0.155 = 144 → clamped to 132, so 132 is the real constant for common devices and the proportion almost never bites — which a reader can't tell without doing the arithmetic.
7. Doc nit (ADR 0004). The edited sentence reads "Argent's ax-service can describe simulator screens that XCTest recursive snapshots and typed XCUIElementQuery enumeration can degrade to no useful child nodes" — the relative clause got mangled (screens don't degrade to no child nodes; the snapshots do). Restore a clean two-clause form.
Verdict: the abstraction work (trait + rename) is a real improvement and I'd keep it as-is. But #1 and #2 are genuine behavior concerns for the physical-device case this PR is explicitly targeting (#1156), and they pull in opposite directions on the same XCTest-channel question — I'd want those two reconciled (and #3 confirmed as intended) before merge. #4–#7 are straightforward cleanups.
Review finding: back --in-app can now report success without finding an in-app back control. In tapTopLeadingNavigationFallback, any delivered synthesized or fallback coordinate tap returns true, so tapInAppBackControl/backInApp reports success on any iOS screen with a valid window frame, including root screens or headers where the top-left tap does nothing or hits a non-navigation control. That regresses the existing contract where absence of an in-app control returned in-app back control is not available. Please gate this fallback on an observed top-band candidate or verify that navigation/state changed before returning success; otherwise keep returning false when no control can be proved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1156.
Reframes iOS snapshot recovery around the XCTest accessibility channel instead of a simulator-only tree backend penalty. Regular snapshots now avoid all XCTest-backed tiers after slow or timed-out XCTest accessibility work and derive recovery from non-XCTest backend traits, while raw diagnostics keep tree-first behavior.
Improves
back --in-appby trying native navigation buttons, then semantic top-band Back/Close/Cancel controls, before falling back to a bounded top-leading tap. This avoids false-positive coordinate success on React Native custom search headers.Validation
pnpm buildpnpm build:xcuitestgit diff --checkthymikee-iphonewith Bluesky: snapshot Explore, focus search by ref, typeagent-device, runback --in-app, and confirm the follow-up snapshot returned to Explore.