Problem
Some iOS Simulator app screens are visible and interactable, but XCTest cannot serialize or enumerate their accessibility tree reliably.
Observed failure mode:
- full XCTest snapshot can fail with
kAXErrorIllegalArgument
- compact interactive XCTest snapshot can degrade to only the synthetic application root
- retrying the same XCTest APIs tends to make the command slower rather than recover semantic UI nodes
The current bounded XCTest behavior is correct for avoiding hangs and false confidence, but it still leaves simulator sessions without semantic snapshot coverage for this class of apps.
Proposal
Add an iOS Simulator-only AX snapshot backend/fallback that does not depend on XCUIElement.snapshot().
Expected shape:
- read the Simulator accessibility surface through host-side Simulator/macOS accessibility APIs, similar in role to idb's simulator accessibility commands
- normalize output into the existing
SnapshotNode model
- keep XCTest as the default runner for actions, physical devices, and normal working snapshots
- use the AX backend as a simulator fallback when XCTest reports
IOS_AX_SNAPSHOT_FAILED or compact interactive output is sparse/root-only
- make fallback choice observable in diagnostics and snapshot warnings
Requirements
- simulator-only; do not imply physical-device support
- deterministic Simulator window/device/app-root selection
- bounded traversal with per-node failure tolerance
- stable frame normalization into simulated device coordinates
- role/label/value/identifier/selected/enabled/hittable mapping compatible with existing selectors where possible
- clear degraded/confidence metadata when root selection or traversal is partial
- no silent fallback that hides XCTest failures without diagnostics
Validation
Add coverage for:
- Settings app snapshot shape and actionable rows
- React Navigation example snapshots and common interactions
- a fixture or repro screen where XCTest snapshot fails or compact output is root-only
- modals, keyboard, scroll views, and multiple Simulator windows/devices
- fallback diagnostics and no-fallback behavior on physical devices
Notes
This should be a separate follow-up to the current bounded XCTest work. The XCTest implementation should remain honest and fast when it cannot inspect a tree; this fallback is for regaining simulator semantic coverage through a different backend.
Problem
Some iOS Simulator app screens are visible and interactable, but XCTest cannot serialize or enumerate their accessibility tree reliably.
Observed failure mode:
kAXErrorIllegalArgumentThe current bounded XCTest behavior is correct for avoiding hangs and false confidence, but it still leaves simulator sessions without semantic snapshot coverage for this class of apps.
Proposal
Add an iOS Simulator-only AX snapshot backend/fallback that does not depend on
XCUIElement.snapshot().Expected shape:
SnapshotNodemodelIOS_AX_SNAPSHOT_FAILEDor compact interactive output is sparse/root-onlyRequirements
Validation
Add coverage for:
Notes
This should be a separate follow-up to the current bounded XCTest work. The XCTest implementation should remain honest and fast when it cannot inspect a tree; this fallback is for regaining simulator semantic coverage through a different backend.