Skip to content

test(e2e): Evaluate replay assertion on Android - #6684

Open
antonis wants to merge 1 commit into
test/restore-replay-e2e-assertionfrom
test/replay-e2e-assertion-android
Open

test(e2e): Evaluate replay assertion on Android#6684
antonis wants to merge 1 commit into
test/restore-replay-e2e-assertionfrom
test/replay-e2e-assertion-android

Conversation

@antonis

@antonis antonis commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Note

Stacked on top of #6683 (iOS replay assertion). Review/merge that first; this PR's base auto-retargets to main once it lands. The diff below is only the Android delta.

📢 Type of change

  • Bugfix

📜 Description

Extends the captureReplay e2e test to run the buffer-priming step and the assertReplay check on Android, not just iOS — the second half of #6082 ("evaluate adding an equivalent Android check").

  • maestro/captureReplay.yml — remove the platform: iOS gates so priming + assertReplay run on both platforms. The Replay Ping primer (testID='replayPing') is already platform-agnostic (RN testID maps to a resource-id Maestro can tap on Android).
  • maestro/utils/assertReplay.yml — relax the codec assertion from the exact ftypmp42 major brand to a valid-MP4-container check (the ftyp box at byte offset 4). iOS (AVAssetWriter) emits mp42; Android (MediaMuxer) can emit a different brand such as isom for the same valid MP4, so the exact-brand check would false-fail on Android. iOS still passes (ftypmp42 starts with ftyp).

💡 Motivation and Context

Second half of #6082

💚 How did you test it?

CI

📝 Checklist

  • I added tests to verify changes.
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled.
  • I updated the docs if needed.
  • I updated the wizard if needed.
  • All tests passing.
  • Public API changes reviewed by another Mobile SDK team member or implemented according to the develop docs spec.
  • No breaking changes.

🔮 Next steps

Runs the captureReplay buffer-priming and assertReplay check on Android in
addition to iOS. Android has never asserted replays in CI (see #4277,
"android doesn't seem to capture replays in CI"); this re-evaluates that
now that deterministic buffer priming lands the assertion reliably on iOS.

Relax the codec assertion from an exact `ftypmp42` major brand to a valid
MP4 container check (the "ftyp" box), since iOS (AVAssetWriter) and Android
(MediaMuxer) can emit different major brands for the same valid MP4.

The Android E2E result on this PR is the deliverable: it empirically shows
whether Android now captures a replay in CI. If red, keep the assertion
iOS-only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@antonis antonis added the ready-to-merge Triggers the full CI test suite label Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Semver Impact of This PR

None (no version bump detected)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


  • test(e2e): Evaluate replay assertion on Android by antonis in #6684
  • test(e2e): Restore iOS replay assertion in captureReplay test by antonis in #6683
  • chore(replay): Mark mobileReplayIntegration as stable by antonis in #6679
  • feat(sample): Add tvOS sample app by antonis in #6677
  • fix(e2e): Pin json gem < 3.0 for RN < 0.72 iOS builds by antonis in #6678
  • feat(ios): Expose enableMemoryIntrospection option by antonis in #6674
  • feat(android): Add anrProfilingSampleRate option by antonis in #6673
  • ci: Unpin Android E2E emulator build by antonis in #6672
  • ref(ios): Remove deprecated private SDK API usage by philprime in #6647
  • chore(deps): update Cocoa SDK to v9.27.0 by github-actions in #6670
  • chore(deps): update Sentry Android Gradle Plugin to v6.21.0 by github-actions in #6671
  • chore(deps): Bump fast-uri to ^3.1.6 to resolve security alerts by antonis in #6662
  • chore(deps): Bump qs to ^6.16.0 to resolve security alerts by antonis in #6663
  • chore(deps): Bump @xmldom/xmldom pins to patched versions to resolve security alerts by antonis in #6664
  • docs: Revamp root and nested AGENTS.md by antonis in #6666

🤖 This preview updates automatically when you update the PR.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 3853.41 ms 1227.88 ms -2625.53 ms
Size 5.15 MiB 6.88 MiB 1.73 MiB

Baseline results on branch: test/restore-replay-e2e-assertion

Startup times

Revision Plain With Sentry Diff
0e1677c+dirty 3858.71 ms 1234.98 ms -2623.73 ms

App size

Revision Plain With Sentry Diff
0e1677c+dirty 5.15 MiB 6.88 MiB 1.73 MiB

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

iOS (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 3863.08 ms 1234.05 ms -2629.03 ms
Size 5.15 MiB 6.88 MiB 1.73 MiB

Baseline results on branch: test/restore-replay-e2e-assertion

Startup times

Revision Plain With Sentry Diff
0e1677c+dirty 3857.73 ms 1232.65 ms -2625.08 ms

App size

Revision Plain With Sentry Diff
0e1677c+dirty 5.15 MiB 6.88 MiB 1.73 MiB

@antonis
antonis marked this pull request as ready for review September 7, 2026 15:05
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 423.59 ms 444.21 ms 20.62 ms
Size 50.56 MiB 56.46 MiB 5.90 MiB

Baseline results on branch: test/restore-replay-e2e-assertion

Startup times

Revision Plain With Sentry Diff
0e1677c+dirty 441.42 ms 462.96 ms 21.54 ms

App size

Revision Plain With Sentry Diff
0e1677c+dirty 50.56 MiB 56.46 MiB 5.90 MiB

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 437.49 ms 459.60 ms 22.11 ms
Size 50.56 MiB 56.46 MiB 5.90 MiB

Baseline results on branch: test/restore-replay-e2e-assertion

Startup times

Revision Plain With Sentry Diff
0e1677c+dirty 527.08 ms 566.04 ms 38.96 ms

App size

Revision Plain With Sentry Diff
0e1677c+dirty 50.56 MiB 56.46 MiB 5.90 MiB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Triggers the full CI test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant