[NoQA] Add Android emulator smoke workflow (agent-device · Phase 0)#89896
Draft
rustam-callstack wants to merge 6 commits into
Draft
[NoQA] Add Android emulator smoke workflow (agent-device · Phase 0)#89896rustam-callstack wants to merge 6 commits into
rustam-callstack wants to merge 6 commits into
Conversation
Phase-0 build-health canary: pulls the developmentDebug APK from Rock's S3 cache, boots a Pixel 8 / API 35 emulator on a Blacksmith Docker runner, installs the APK, launches via agent-device, and uploads a landing-screen screenshot, accessibility tree, foreground-app dump, logcat, and a short cold-start MP4. Does NOT log in or run a tab tour — that's Phase 1, deferred until an Onyx-seed login bypass exists. Triggered on PR open/synchronize (with the same paths-ignore / branches-ignore / OSBotify-skip guards as reassurePerformanceTests.yml) and via workflow_dispatch. Three cost guards: concurrency cancel-in-progress keyed by ref, 35-min timeout, paths-ignore for docs/tests/.github/.sh. Files: - .github/workflows/smokeAndroid.yml - .github/scripts/agent-device-smoke.sh
added 4 commits
May 7, 2026 15:22
Replaces the bare sleep 8 (which captured the splash on slower runners) with a snapshot poll up to 120s waiting for the email text-field to appear. After it lands: - screenshot landing.png + snapshot-signin.txt - agent-device fill 'label="Phone or email"' rustam.zeinalov@callstack.com - agent-device press 'label="Continue"' - snapshot poll up to 30s waiting for the magic-code field - screenshot post-continue.png + snapshot-magic-code.txt Hard fails on either timeout. The flow stops short of typing the magic code itself: that 6-cell composite input rejects every programmatic input we tested. Solving it is Phase 1 (Onyx-seed login bypass).
Hermes-engine's configureCMakeRelease[x86_64] task pins CMake 3.30.5 exactly. Pre-installed runner images don't always ship it (GitHub bumps ubuntu-latest periodically; Blacksmith image contents may also shift). If Rock's S3 fingerprint misses and we fall back to local Gradle, the build dies with [CXX1300] CMake '3.30.5' was not found. sdkmanager --install "cmake;3.30.5" lands the binaries under $ANDROID_HOME/cmake/3.30.5/bin where Gradle auto-detects them. Idempotent on cached runners; ~30s overhead on cold.
Free ubuntu-latest is significantly slower than the 4-core Blacksmith runner. The 120s budget covers it on the upstream path but leaves the fork-test capturing only the splash screen at timeout (verified in artifacts: green E logo, no SignIn UI). Also bumped post-Continue timeout to 60s for symmetry; the magic-code screen typically appears within 5s but a slow runner could need more.
Last run hit a Diagnostic at fill, because Android's text-field label is "Phone or email," (with trailing comma) while iOS has no comma, and agent-device's selector form does exact-match. Cross-platform fix: grep the @e ref out of the snapshot we already captured (refs are stable inside one session as long as we don't re-snapshot in between) and act on it directly. Also bumped SIGNIN_LOAD_TIMEOUT 300 -> 360. Last run reached SignIn at 291s, dangerously close to the previous 300s ceiling.
This was referenced May 11, 2026
Bash has no native block-comment, but `: <<'COMMENT' ... COMMENT` is the canonical idiom (ShellCheck recognizes it). Mirrors the TS PR's recent comment-style change for consistency. Single-line `#` comments left as-is — no clean equivalent. Shebang and ShellCheck directives preserved. `bash -n` syntax check still passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Explanation of Change
Adds a Phase-0 build-health canary that runs on every PR + on manual dispatch:
developmentDebugAPK from Rock's S3 cache (npx rock build:android --variant developmentDebug) — falls back to a local Gradle build if the fingerprint misses.google_apis/ x86_64 emulator onblacksmith-4vcpu-ubuntu-2404viareactivecircus/android-emulator-runner@v2with the standard two-stage AVD-cache pattern.script:block, the new.github/scripts/agent-device-smoke.sh:adb logcatintoartifacts/logcat.txtadb installs the APK andadb reverse tcp:8081 tcp:8081/statusreachingpackager-status:runningagent-device open … --relaunch(pinned toagent-device@0.14.7)artifacts/withif: always()so failures are debuggable.Phase 0 explicitly does not log in or run a tab tour — the magic-code 6-cell input rejects all programmatic input we tested locally (
agent-device fill,adb shell input text, paste). The full tab-tour smoke is deferred to a separate Phase 1 change which would add a build-variant + Onyx-state seed delivery + 1Password rotation. This PR is the workflow harness only.Cost guards
concurrency.cancel-in-progresskeyed by ref: newer pushes to the same PR cancel older smoke runstimeout-minutes: 35— hard ceilingpaths-ignorefordocs/**,help/**,.github/**,contributingGuides/**,tests/**,**.md,**.sh(matchesreassurePerformanceTests.yml)if: github.actor != 'OSBotify'(matchesreassurePerformanceTests.yml)Required secrets
All already used by other workflows in this repo:
OS_BOTIFY_TOKEN,MAPBOX_SDK_DOWNLOAD_TOKEN,AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY.Fixed Issues
$
PROPOSAL:
No tracking issue exists for this work yet — opening as a draft so it can be discussed before going through formal Bug Zero / proposal flow. Happy to move it to ready-for-review and link an issue once direction is confirmed.
Tests
This PR introduces a CI workflow only — no runtime app code changes. Verification is exercising the workflow itself.
workflow_dispatch).blacksmith-4vcpu-ubuntu-2404./dev/kvmexists and is writable).Downloaded cached build from S3 (…)and that an APK lands atandroid/app/build/outputs/apk/development/debug/*.apk.artifacts/landing.pngshowing the Expensify SignIn screenartifacts/snapshot.txtcontaining an accessibility tree with refs like@e<n> [text-field] \"Phone or email\" [editable]artifacts/appstate.txtcontainingForeground app: com.expensify.chat.devartifacts/logcat.txt(non-empty)artifacts/cold-start.mp4(≥ 1 KB)smoke-android-<run_id>-<run_attempt>uploads successfully (visible in the run summary).src/) and confirm the workflow auto-fires.cancel-in-progresscancels the prior run.Failure-mode probe (optional):
Toggle
emulator-options: -accel offto force the boot to fail; confirmtimeout-minuteskicks in and the Upload artifacts step still runs (if: always()).Verify that no errors appear in the JS console
Offline tests
N/A — this PR adds CI infrastructure only. No app runtime is changed.
QA Steps
N/A — files under
.github/are not shipped to staging or production.PR Author Checklist
I linked the correct issue in the `### Fixed Issues` section above
No tracking issue exists yet; opened as a draft to discuss direction first.
I wrote clear testing steps that cover the changes made in this PR
I included screenshots or videos for tests on all platforms — N/A, CI workflow change only
I ran the tests on all platforms — N/A, CI workflow change only:
I verified there are no console errors
I followed proper code patterns
I followed the guidelines as stated in the Review Guidelines
I tested other components that can be impacted by my changes — N/A
I verified all code is DRY
I verified any variables that can be defined as constants are
I verified that if a function's arguments changed that all usages have also been updated correctly
If any new file was added I verified that:
Screenshots/Videos
Android: Native
N/A — CI workflow change only. The workflow itself produces
landing.png+cold-start.mp4artifacts that demonstrate end-to-end behavior; those will be attached after the first successful run from this branch.Android: mWeb Chrome
N/A
iOS: Native
N/A — Phase 0 is Android only (see Phase 1 plan in the workflow file's header comment).
iOS: mWeb Safari
N/A
MacOS: Chrome / Safari
N/A