test: add Android full emulator coverage

[thymikee]

Summary

Extends the Android emulator coverage introduced in #1482 with a full nightly tier while keeping the required smoke tier focused. The full suite adds fixture-observed coverage for:

• runtime microphone permission accept, deny, reset, revoke, and recovery
• typed push broadcasts delivered to an actual Android receiver
• IME-owned input recovery
• helper-backed swipe, scroll, pan, fling, pinch, rotate, and transform replay
• performance metrics, logcat, traces, screen recordings, batch results, paged events, artifact inventory/download, and JUnit replay reports
• retry-free long-list navigation and fixture replay suites

The coverage report now records the executed tier and a manifest-derived classification rollup, and the nightly lane requires the trusted native fixture produced for the exact head.

Bugs exposed and fixed

• trace stop wrote a file but did not publish it as a daemon-managed trace-log, so it was absent from artifact inventory and could not be downloaded through the artifact endpoint.
• The MCP output schema did not advertise the stopped trace's artifact payload, leaving schema-driven clients unaware of the downloadable result.
• The Android nightly lane did not guarantee packaged snapshot/IME helpers before exercising helper-backed gestures.
• The nightly lane could silently rebuild a missing fixture instead of proving reuse of the trusted exact-head native artifact.
• The test fixture did not declare Android audio recording permission, preventing real microphone permission recovery coverage.
• The fixture's Expo module registration used the wrong Android class name, breaking Release compilation.
• The persisted Android coverage report omitted its classification summary and executed tier.

Builds on merged PR #1482. Scope spans Android E2E scenarios, the Android test fixture, trace artifact publication, and CI orchestration (31 files).

Validation

• Android coverage/report contract tests pass, including artifact-level report persistence.
• Root and Expo test-app typechecks pass.
• Static, integration, coverage, layering, Fallow, packaging, Linux, macOS, and CodeQL checks pass on the current head.
• Android Release and exact-head Android/iOS smoke remain the authoritative native/device validation.

[thymikee]

PR #1484 is not ready at exact head 8538b7b.

• P1 — stacked-base blocker. The PR is based on test(android): add comprehensive emulator E2E coverage #1482 head d8de04d, while test(android): add comprehensive emulator E2E coverage #1482 has advanced; GitHub reports this branch conflicted. This head has no PR checks, so there is no exact-head CI or inspectable full-emulator evidence. Rebase onto test(android): add comprehensive emulator E2E coverage #1482’s accepted head, resolve the dependency first, and rerun the required checks plus the Android full-emulator lane.
• P2 — trace MCP schema drift. TraceCommandResult and record-trace.ts now require/return artifacts for trace: "stopped", but src/mcp/command-output-schemas.ts still advertises only trace and outPath. Schema-driven clients will not discover the downloadable trace-log artifact. Add the standard artifact array to the stopped-trace schema and a focused MCP output-schema assertion.

No ready-for-human label while the branch is conflicted, lacks exact-head evidence, and carries this contract drift.

[thymikee]

Exact-head review found two blockers:

1. Stack/base blocker: this draft is still stacked on a stale test(android): add comprehensive emulator E2E coverage #1482 base and is currently conflicting. It has no exact-head GitHub checks. Rebase onto test(android): add comprehensive emulator E2E coverage #1482’s accepted head, resolve conflicts, then rerun the required checks and full Android emulator evidence before requesting readiness.
2. MCP schema drift: trace stop now returns a required artifacts array with a downloadable trace-log, but the stopped trace branch in src/mcp/command-output-schemas.ts still declares only trace and outPath. Add the standard artifacts schema and a focused schema assertion so schema-driven clients can discover the new response field.

The manifest/evidence machinery is otherwise structurally meaningful, but the expanded live-scenario scope goes beyond #1426’s original guardrail and needs explicit coordination/approval after the stack is repaired.

[thymikee]

Exact head 8538b7b is not ready: it is conflicting against the rewritten #1482 stack, and two gaps remain. (1) trace stop now requires/emits artifacts, but the MCP output schema still advertises only trace and outPath; add required artifact schema coverage. (2) coverage-report.json contains runtime evidence/timing only, but #1426 requires manifest-derived live/contract/capability-denial/known-gap counts; add the rollup and a drift-proof test. Rebase first, then validate the resolved exact head with CI/live emulator evidence.

[thymikee]

Dependency update: #1482 has now merged, while this draft remains on unchanged head 8538b7b and is DIRTY/conflicting with no exact-head checks. Restack/rebase onto accepted main before re-review. The previously posted MCP trace-schema and coverage-rollup findings remain unresolved; no duplicate detail review until the head changes.

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	1.90 MB	1.90 MB	+187 B
JS gzip	609.7 kB	609.7 kB	+80 B
npm tarball	727.4 kB	727.5 kB	+69 B
npm unpacked	2.55 MB	2.55 MB	+218 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	27.1 ms	27.9 ms	+0.8 ms
CLI --help	58.0 ms	57.8 ms	-0.2 ms

Top changed chunks: no changes in the largest emitted chunks.

[thymikee]

Re-review at eacf244: not ready.

1. P1 — Android Release is broken by the new Expo module registration. expo-module.config.json lists bare PushBroadcastLabModule, while the Kotlin class is expo.modules.pushbroadcastlab.PushBroadcastLabModule. Exact-head CI fails :expo:compileReleaseKotlin in generated ExpoModulesPackageList.kt, so no fixture APK or live Android validation can run. Register the fully qualified Android module class, then rerun Android Release and the emulator lanes.
2. P2 — make the coverage-rollup regression test exercise the report artifact. The current test proves only the in-memory summary constant, not that coverage-report.json receives it. Removing classificationSummary from writeCoverageReport would still leave the test green. Write/read a report in the test and assert its counts and tier.

The previously reported stopped-trace MCP artifact-schema issue is resolved. Static, Coverage, Integration, type/layering, Linux, macOS, and CodeQL are green; Android Release is a confirmed PR-owned failure and device smoke is not complete.

[thymikee]

Addressed the current review findings in 84b1416:\n\n- Registered the Android Expo module as expo.modules.pushbroadcastlab.PushBroadcastLabModule, matching the Kotlin declaration used by generated Expo registration.\n- Added an artifact-level regression test that writes and reads coverage-report.json, asserting the manifest-derived classification rollup and executed tier.\n\nFocused validation passed: Android coverage tests (10/10), root typecheck, and Expo test-app typecheck. The exact-head Android Release build is running: https://github.com/callstack/agent-device/actions/runs/30555392709/job/90914617723

[thymikee]

The full-tier coverage is valuable and the production trace-artifact fix is sound, but exact head 84b1416cf is not ready yet.

1. P2 — keep timing diagnostic-only. test/integration/android-emulator-e2e/live-coverage-report.ts:17-34 reintroduces a ten-minute wall-clock assertion into functional coverage. #1482 explicitly removed this because shared emulator load can make a correct journey exceed an arbitrary threshold. The job already has an outer timeout and the report records timings; delete the assertion/import/constant.
2. P2 — do not rediscover the coverage report by traversal order. .github/workflows/android.yml:78-79 and .github/workflows/replays-nightly.yml:142-143 use find ... | tail -n 1. With multiple artifact directories this can select a stale report—the same defect corrected during #1482. The runner already prints the exact report path and timing summary, so remove both lookups and inline Node programs.
3. P2 — reuse completed build infrastructure. .github/workflows/android.yml:75-76 repeats the dedicated CLI build and daemon reset immediately above the emulator action. In the nightly workflow, setup-android-replay-host already restores or packages and verifies both Android helpers, but line 137 repackages them with pnpm build:android. Keep only work that the emulator genuinely requires.

Exact-head Android/iOS smoke and native Release checks are still running. No ready-for-human label until these regressions are removed and the full Android emulator suite passes.

[thymikee]

One additional P2 from the tightening pass: android-emulator-e2e/event-timeline.ts is a formatting-level fork of ios-simulator-e2e/event-timeline.ts, and live-replay-evidence.ts is likewise a near-copy of iOS replay-evidence.ts (~100 lines total). The event paginator is only regression-tested through the iOS copy, so Android can drift while the tests stay green. Move both platform-neutral helpers under test/integration/live-device-e2e/ and have Android/iOS import the shared implementations; keep only retry-count/platform policy at the caller boundary.

[thymikee]

Addressed the latest tightening-pass feedback in 29ab77f:\n\n- Moved event pagination and replay-evidence logic to shared test/integration/live-device-e2e/ helpers; both Android and iOS now import the same paginator, retaining the existing iOS regression oracle. Retry count remains supplied at the platform caller (Android: 0, iOS: configured retries).\n- Removed the Android functional wall-clock assertion; timings remain report diagnostics and the workflow timeout remains the execution bound.\n- Removed both find … | tail -n 1 coverage-report rediscovery blocks and their inline Node programs; the runner already emits the exact report path and timing summary.\n- Removed redundant pre-emulator CLI build/daemon reset and nightly build:android; setup-android-replay-host remains responsible for the verified helper setup.\n\nValidation passed: Android/iOS coverage tests (22/22) and root typecheck. Exact-head CI is running: https://github.com/callstack/agent-device/pull/1484/checks

[thymikee]

Re-review at 29ab77f: the prior Expo registration, persisted coverage-report test, and stopped-trace MCP schema findings are fixed; Android Release and all static/package checks are green.

Not ready: both exact-head device smokes reach the fixture but fail on deterministic selector state—iOS cannot find automation-press after the input-canary scroll, and Android cannot find automation-open-sheet after Home/Recents restoration while the surface remains scrolled. The exact base/main commit 0ee2a861 passed both platform workflows, so this is not a repository-wide infrastructure outage. Make the scenarios explicitly restore/reveal their expected fixture landmark, verify the repacked fixture, and rerun both exact-head smokes. There is also no successful exact-head Android full-tier run yet, so the new live trace/artifact coverage remains unvalidated.

No ready-for-human label while those device checks are red. Residual risk: no separately authorized cross-vendor review was performed.

[thymikee]

Addressed the exact-head device-smoke failures in 9ef5bf1:\n\n- iOS now reveals automation-press through the bounded selector/scroll helper before asserting or pressing it, rather than relying on a fixed 120-pixel scroll.\n- Android explicitly returns the restored Automation route to its top landmark after Home/Recents before waiting for automation-open-sheet.\n\nThe root cause was deterministic fixture scroll state, not platform infrastructure. Focused Android/iOS coverage validation passed (22/22) with root typecheck. Exact-head Android and iOS smoke jobs have restarted: https://github.com/callstack/agent-device/pull/1484/checks