You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This issue is the canonical React Native overlay reliability issue and consolidates #565, #566, and #577.
React Native overlay handling is currently split across several places:
snapshot presentation decides which overlay accessibility nodes to preserve or suppress
snapshot warning output separately detects React Native overlay text
react-native dismiss-overlay separately resolves a safe target and sends an action
screenshot --overlay-refs does not share the same overlay action model
That split makes the agent experience less reliable: collapsed LogBox banners can have close affordances in different positions, failures can leave agents unsure whether to inspect refs or press visible text, and dismissal can continue without a verified post-action view when stricter automation needs certainty.
Goal
Create one React Native overlay analysis model that all overlay-related callers consume. The model should preserve enough fidelity for both user experience and reliability:
overlay kind, such as collapsed LogBox, expanded LogBox, or RedBox
severity and reporting metadata when available
visible evidence nodes used to identify the overlay
safe controls/actions, preferring snapshot refs or selectors over coordinates
explicit marking of coordinate fallback as fallback, not the primary target model
agent-facing warnings and next-command guidance
optional post-action verification state
User experience and reliability requirements
Prefer explicit accessibility targets when a close, dismiss, or minimize control is present.
Never press the warning body or a generic full-screen warning parent as a dismissal shortcut.
Use coordinate fallback only when the layout is confidently recognized as a safe collapsed banner affordance.
Handle non-LTR and non-trailing close targets; do not assume the close affordance is always on the trailing edge.
Keep the default react-native dismiss-overlay fast path: dismiss once, then tell the agent to take a fresh snapshot -i -c before continuing.
Add an explicit stricter verification mode, for example --verify, for automation that must confirm the overlay is gone before continuing.
Report verified: true only after a fresh post-dismiss snapshot no longer detects the overlay.
When dismissal is ambiguous, unsafe, or the overlay remains visible, return a useful warning or failure payload with the next action for the agent.
Failure guidance should point agents to screenshot --overlay-refs rather than pressing the banner body when no safe target is found.
Proposed direction
Model React Native overlays as snapshot analysis artifacts, for example snapshot.overlays[] or snapshot.issues[], with safe actions attached to each detected overlay. Then make these callers consume the same analysis result:
iOS snapshot presentation preserves or suppresses overlay nodes through the overlay analysis module
snapshot warnings are generated from the shared analysis
react-native dismiss-overlay consumes the shared safe-action model
verified dismissal performs a post-action snapshot before reporting final verified success
screenshot --overlay-refs annotates overlay targets consistently with the command's safe-action model
Acceptance criteria
One module owns React Native overlay recognition vocabulary and safe-action selection.
Snapshot presentation, warning generation, react-native dismiss-overlay, and screenshot --overlay-refs do not duplicate overlay heuristics.
Collapsed LogBox dismissal prefers a concrete close or dismiss node from the snapshot when available.
Collapsed LogBox dismissal remains safe for the existing iOS-style banner plus non-trailing or RTL-like close targets.
Coordinate fallback is used only when the layout is confidently recognized as a safe collapsed banner target.
Full-screen warning parents are not exposed as generic app targets.
RedBox behavior still prefers Minimize over Dismiss.
react-native dismiss-overlay keeps the current fast behavior by default and does not claim verified dismissal without a post-action snapshot.
A verification option is available for stricter automation, for example --verify.
Verified success reports verified: true only when a fresh post-dismiss snapshot no longer detects the overlay.
Ambiguous, unsafe, or still-visible overlays preserve a useful failure or warning payload with the next action for the agent.
Failure output tells agents to use screenshot --overlay-refs rather than pressing the banner body when no safe target is found.
Tests cover collapsed LogBox, expanded LogBox, RedBox, no-overlay app copy, existing iOS-style collapsed banner behavior, non-trailing or RTL-like close target behavior, verified success, and still-visible overlay behavior.
Scope guard
This is a React Native overlay reliability issue, not a generic overlay framework for every platform or native modal type. Keep changes focused on RN overlay recognition, safe dismissal actions, user-facing recovery guidance, and verification.
Consolidated scope
This issue is the canonical React Native overlay reliability issue and consolidates #565, #566, and #577.
React Native overlay handling is currently split across several places:
react-native dismiss-overlayseparately resolves a safe target and sends an actionscreenshot --overlay-refsdoes not share the same overlay action modelThat split makes the agent experience less reliable: collapsed LogBox banners can have close affordances in different positions, failures can leave agents unsure whether to inspect refs or press visible text, and dismissal can continue without a verified post-action view when stricter automation needs certainty.
Goal
Create one React Native overlay analysis model that all overlay-related callers consume. The model should preserve enough fidelity for both user experience and reliability:
User experience and reliability requirements
react-native dismiss-overlayfast path: dismiss once, then tell the agent to take a freshsnapshot -i -cbefore continuing.--verify, for automation that must confirm the overlay is gone before continuing.verified: trueonly after a fresh post-dismiss snapshot no longer detects the overlay.screenshot --overlay-refsrather than pressing the banner body when no safe target is found.Proposed direction
Model React Native overlays as snapshot analysis artifacts, for example
snapshot.overlays[]orsnapshot.issues[], with safe actions attached to each detected overlay. Then make these callers consume the same analysis result:react-native dismiss-overlayconsumes the shared safe-action modelscreenshot --overlay-refsannotates overlay targets consistently with the command's safe-action modelAcceptance criteria
react-native dismiss-overlay, andscreenshot --overlay-refsdo not duplicate overlay heuristics.MinimizeoverDismiss.react-native dismiss-overlaykeeps the current fast behavior by default and does not claim verified dismissal without a post-action snapshot.--verify.verified: trueonly when a fresh post-dismiss snapshot no longer detects the overlay.screenshot --overlay-refsrather than pressing the banner body when no safe target is found.Scope guard
This is a React Native overlay reliability issue, not a generic overlay framework for every platform or native modal type. Keep changes focused on RN overlay recognition, safe dismissal actions, user-facing recovery guidance, and verification.
Blocked by
None - can start immediately