Skip to content

fix: handle Android IME overlays in snapshots#1157

Merged
thymikee merged 4 commits into
mainfrom
fix/android-maestro-ime-snapshot
Jul 8, 2026
Merged

fix: handle Android IME overlays in snapshots#1157
thymikee merged 4 commits into
mainfrom
fix/android-maestro-ime-snapshot

Conversation

@thymikee

@thymikee thymikee commented Jul 8, 2026

Copy link
Copy Markdown
Member

Summary

Fix Android snapshot handling when Gboard/IME overlays are present during Maestro-compatible React Native flows.

This moves IME overlay classification into shared Android/core snapshot handling, bounds Android Maestro verification retries after native wait succeeds, and avoids pathological occlusion scans by treating large IME subtrees as one overlay root.

Validation

pnpm exec vitest run src/daemon/handlers/tests/snapshot-capture.test.ts src/compat/maestro/tests/runtime-assertions.test.ts src/compat/maestro/tests/runtime-interactions.test.ts src/platforms/android/tests/snapshot-content-recovery.test.ts
pnpm format
pnpm build
pnpm check:quick
pnpm build:android-snapshot-helper
pnpm package:android-snapshot-helper:npm

Manual Android: React Navigation focused 4-flow Maestro set passed on Pixel_9_Pro_XL with Metro on port 8082: stack-prevent-remove, drawer-master-detail, showcase-material-top-tabs, native-stack-prevent-remove.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.6 MB 1.6 MB +1.7 kB
JS gzip 507.7 kB 508.2 kB +464 B
npm tarball 610.1 kB 610.6 kB +502 B
npm unpacked 2.2 MB 2.2 MB +1.7 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 27.4 ms 29.3 ms +2.0 ms
CLI --help 58.8 ms 56.7 ms -2.0 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/android-input-ownership.js +17.7 kB +6.3 kB
dist/src/internal/daemon.js +778 B +169 B
dist/src/tv-remote.js +502 B +108 B
dist/src/session.js +109 B +39 B
dist/src/apps.js +17 B +10 B

import { classifyAndroidInputOwnership } from './input-ownership.ts';

const GBOARD_PACKAGE = 'com.google.android.inputmethod.latin';
const GBOARD_HANDWRITING_TITLE = 'Try out your stylus';

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detection is gated on the English title string 'Try out your stylus' — both hasAndroidGboardHandwritingTutorial and findAndroidGboardHandwritingTutorialCancel bail unless a node's label/value matches it exactly.

On any non-English device locale the Gboard tutorial title is localized, so the whole fix silently no-ops: hasAndroidGboardHandwritingTutorial returns false → no preflight dismissal, no occlusion handling, and the flow falls back to spending the full native-wait budget behind the keyboard again.

The cancel node is already found by the locale-stable android:id/closeButton id (with 'Cancel' only as a fallback). Consider keying tutorial detection off that same id (a Gboard-owned closeButton in the IME subtree) rather than the translated title, so the dismissal works regardless of locale.

@thymikee thymikee force-pushed the fix/android-maestro-ime-snapshot branch from 1a5e59e to 6799fa7 Compare July 8, 2026 16:14
@thymikee

thymikee commented Jul 8, 2026

Copy link
Copy Markdown
Member Author

Reviewed current head 6799fa7 against the Android IME/Maestro snapshot changes. The implementation moves IME ownership into shared Android snapshot handling, bounds the post-native Maestro visible retry, and treats large IME subtrees as one overlay root so occlusion scans stay bounded. Focused tests cover helper recovery, overlay occlusion, and Maestro assertion retry behavior; CI is green across 21/21 checks. The PR also includes manual Pixel_9_Pro_XL React Navigation flow evidence for the device-facing path. No actionable blockers found.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Jul 8, 2026
@thymikee thymikee merged commit f21727d into main Jul 8, 2026
21 checks passed
@thymikee thymikee deleted the fix/android-maestro-ime-snapshot branch July 8, 2026 16:29
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-08 16:30 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant