feat(app): bundle device e2e artifacts

[lalalune]

Summary

• add a shared device-e2e bundle assembler that writes inline/, logs/, raw/, reports/, summary.json, and junit.xml
• make android-e2e.mjs and ios-e2e.mjs accept --output <dir> and always finalize the bundle on normal pass/fail paths
• capture Android route-coverage screenrecord plus final screenshot/logcat into the bundle
• capture iOS simulator final screenshot/video/log into the bundle when a simulator is available
• point Android Playwright JUnit/JSON/HTML output into the bundle and let Android specs honor ELIZA_ANDROID_ARTIFACT_DIR instead of writing under .github/issue-evidence during runner-driven runs
• add deterministic bundle unit coverage for output parsing, summary/JUnit writing, inline artifact copies, and failed-step reporting

Relates to #14336.

Stacked on #14483 because it builds on the device lease/freshness runner work.

Validation

• node --check packages/app/scripts/lib/device-e2e-bundle.mjs && node --check packages/app/scripts/android-e2e.mjs && node --check packages/app/scripts/ios-e2e.mjs && node --check packages/app/scripts/ios-e2e-lib.mjs && git diff --check
• bunx @biomejs/biome check packages/app/scripts/lib/device-e2e-bundle.mjs packages/app/scripts/device-e2e-bundle.test.mjs packages/app/scripts/android-e2e.mjs packages/app/scripts/ios-e2e.mjs packages/app/scripts/ios-e2e-lib.mjs packages/app/scripts/ios-e2e-lib.test.mjs packages/app/playwright.android.config.ts packages/app/test/android/onboarding-to-home.android.spec.ts packages/app/test/android/lifecycle.android.spec.ts packages/app/test/android/touch-gesture.android.spec.ts packages/app/test/android/view-runtime-soak.android.spec.ts packages/app/test/android/lifecycle-reboot.android.spec.ts packages/app/test/android/sleep-wake.android.spec.ts packages/app/test/android/launcher-gesture-loop.android.spec.ts packages/app/test/android/native-plugin-view-smoke.android.spec.ts
• bunx vitest run --config packages/app/vitest.config.ts packages/app/scripts/device-e2e-bundle.test.mjs packages/app/scripts/ios-e2e-lib.test.mjs
• bunx playwright test --config packages/app/playwright.android.config.ts --list (74 tests listed)

Failure-path smoke

Linux host cannot run xcrun simctl, but the iOS runner now still finalizes its bundle before exiting non-zero:

[ios-e2e] plan: build -> auth -> local-chat
[ios-e2e] bundle: /tmp/eliza-ios-bundle-smoke
[ios-e2e] FAILED: iOS e2e requires macOS (xcrun simctl).

The produced summary.json included:

{
  "lane": "ios-sim",
  "steps": [
    {
      "name": "boot iOS Simulator",
      "status": "failed",
      "error": "iOS e2e requires macOS (xcrun simctl)."
    }
  ],
  "artifacts": [
    {
      "kind": "junit",
      "path": "junit.xml"
    }
  ],
  "result": "failed"
}

Evidence still needed before merge

• One full Android route-coverage run with the generated bundle attached inline (inline/*.mp4, inline/*.jpg, summary.json, junit.xml).
• One macOS iOS-simulator run with the generated bundle attached inline.
• Manual GitHub drag/drop check that generated inline/*.mp4 renders inline.

[lalalune]

Reviewed by inspection (own-account, commenting). Clean, self-contained additive tooling that closes #14336: a per-device-e2e artifact bundler (lib/device-e2e-bundle.mjs) assembling one run dir (inline/logs/raw/reports + summary.json + junit.xml) that survives non-zero exits and prepares inline-friendly evidence (PNG→JPG, MOV→MP4). error-policy clean — runBundledCommand marks the step failed AND rethrows on non-zero/signal; the filesystem-probe catches are genuine J3 invalid-signals; conversion failures go to warnings, never faked success. The -122 is internal refactor of the android/ios runners (same command sequence, now bundle-wrapped) + backward-compatible env-gated artifact paths — no shared build/CI/dev orchestration touched. Device-free vitest present.

Two blockers before merge: (1) base is not develop — it's stacked on fix/14339-device-lease-lock (#14483), which is itself on the #14480→#14475 stack; retarget to develop after that stack lands, and re-verify the android/ios runner refactor reconciles once #14483's lease code is on develop (this PR's deletions remove lease lines it introduced — coherent within the stack, expect friction post-rebase). (2) Your own body flags device-run evidence still owed — no real Android/iOS-sim bundle run, no inline-.mp4 render check. So even post-retarget it's NEEDS-DEVICE-VERIFY for the actual bundle output. Good to go once rebased onto develop + a real bundle run is attached.

[lalalune]

[coordination] This is the canonical #14336 bundle (creates device-e2e-bundle.mjs, wires the runners + Playwright reporters) — thanks. I've retargeted #14505 down to a single additive lib, packages/app/scripts/lib/device-e2e-visual-qa.mjs, and it does not touch any runner or the bundle assembler, so it composes with this PR instead of colliding.

attachVisualQa({ bundleDir, screenshotsDir, expectations }) consumes analyzeScreenshot from lib/visual-qa.mjs (#14433) and, for every screenshot in a produced bundle, writes an OCR + dominant-palette + brand-colour + expectation report as <image>.visual-qa.json beside the pixels, plus a root visual-qa.json verdict roll-up. It's the visual-qa-consumption half of #14336 that this bundle PR doesn't cover.

Offer: happy to fold this straight into your bundle finalize step — a single await attachVisualQa({ bundleDir, screenshotsDir }) call after the screenshots land — so it ships as one coherent bundle feature rather than a separate follow-up. Let me know if you'd rather I open a small PR against your branch or you'd prefer to cherry-pick the lib. (Note: #14502 owns #14338 forensics separately — see my comment there.)

[lalalune]

Rebased this branch onto the updated origin/fix/14339-device-lease-lock, skipping duplicate stack-base commits already present in #14483/#14475. Current diff is one commit over #14483: feat(app): bundle device e2e artifacts.

Verification from /private/tmp/codex-14494-device-bundle:

• bun run --cwd packages/app test -- scripts/device-e2e-bundle.test.mjs scripts/ios-e2e-lib.test.mjs -> 2 files / 46 tests passed
• bunx biome check packages/app/scripts/lib/device-e2e-bundle.mjs packages/app/scripts/device-e2e-bundle.test.mjs packages/app/scripts/android-e2e.mjs packages/app/scripts/ios-e2e.mjs packages/app/playwright.android.config.ts packages/app/test/android/launcher-gesture-loop.android.spec.ts packages/app/test/android/lifecycle-reboot.android.spec.ts packages/app/test/android/lifecycle.android.spec.ts packages/app/test/android/native-plugin-view-smoke.android.spec.ts packages/app/test/android/onboarding-to-home.android.spec.ts packages/app/test/android/sleep-wake.android.spec.ts packages/app/test/android/touch-gesture.android.spec.ts packages/app/test/android/view-runtime-soak.android.spec.ts -> passed
• node --check packages/app/scripts/lib/device-e2e-bundle.mjs && node --check packages/app/scripts/android-e2e.mjs && node --check packages/app/scripts/ios-e2e.mjs -> passed
• git diff --check origin/fix/14339-device-lease-lock...HEAD -> passed

[lalalune]

Rebased again onto the latest origin/fix/14339-device-lease-lock after #14483 retargeted to current develop. Conflict was limited to packages/app/.gitignore; kept both capture-output/ and device-e2e-output/.

Verification from /private/tmp/codex-14494-device-bundle:

• bun run --cwd packages/app test -- scripts/device-e2e-bundle.test.mjs scripts/ios-e2e-lib.test.mjs -> 2 files / 46 tests passed
• bunx biome check ... on the touched app bundle files -> passed
• node --check packages/app/scripts/lib/device-e2e-bundle.mjs && node --check packages/app/scripts/android-e2e.mjs && node --check packages/app/scripts/ios-e2e.mjs -> passed
• git diff --check origin/fix/14339-device-lease-lock...HEAD -> passed

[claude]

Claude encountered an error —— View job

---

I'll analyze this and get back to you.