Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port CircleCI Artifact downloads to speed up release testing to 0.72 #38553

Merged
merged 1 commit into from
Jul 24, 2023

Conversation

cipolleschi
Copy link
Contributor

@cipolleschi cipolleschi commented Jul 21, 2023

Summary:

Cherry picking f6197cd to speedup release testing.

Changelog:

[internal] - cherry picking improved testing script

Test Plan:

tested locally

@facebook-github-bot facebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner labels Jul 21, 2023
@cipolleschi cipolleschi changed the base branch from main to 0.72-stable July 21, 2023 15:17
@cipolleschi cipolleschi force-pushed the cipolleschi/port-download-ci-to-0.72 branch from bfeb117 to f73b646 Compare July 21, 2023 15:22
@analysis-bot
Copy link

analysis-bot commented Jul 21, 2023

Platform Engine Arch Size (bytes) Diff
android hermes arm64-v8a 8,486,147 -357,135
android hermes armeabi-v7a 7,804,208 -348,212
android hermes x86 8,956,189 -392,856
android hermes x86_64 8,817,156 -374,643
android jsc arm64-v8a 9,147,136 -308,872
android jsc armeabi-v7a 8,336,616 -300,524
android jsc x86 9,200,204 -338,889
android jsc x86_64 9,458,631 -323,755

Base commit: 16fe8da
Branch: main

Summary:
Testing releases takes a lot of time because we have to build locally several configurations.
However, the artifacts that we build locally are also built in CI.

The goal of this PR is to implement a mechanism to download those artifacts from the CI instead of build locally, so that testing the release locally can take much less time.

As an example, the full test cycle can take more than 2 hours given that we need to repackage and rebuilt the app from the template.

My plan is to add a table with the time saved once the PR is done

- [x] Download Hermes tarball for RNTester iOS
- [x] Download Hermes APK for RNTester Android
- [x] Download JSC APK for RNTester Android
- [x] Download Packaged version of React Native to create a new app
- [x] Use the downloaded React Native to initialize an app from the template
- [x] Download Maven Local prebuilt in CI and use it for Template Android app

| Setup | Before [s] | After [s] | Notes |
| --- | --- | --- | --- |
| iOS RNTester Hermes | 339.68 | 194.86 | Time saved by downloading Hermes rather then building it |
| iOS RNTester JSC | 129.80 | 123.35 | Not significant, expected as this workflow did not change
| Android RNTester Hermes | 1188.82 | 5.28 | Huge improvement: we download the APK rather then build |
| Android RNTester JSC | 103.10  | 6.28 | Huge improvement: we download the APK rather then build  |
| Creating the RNTestProject | 2074.82  | 191.16 | We download Maven, the packaged version of RN and Hermes instead of building from scratch |

[Internal] - Speed up Release testing by downloading the CircleCI artifacts

Pull Request resolved: #37971

Test Plan: - Tested the script locally

Reviewed By: cortinico, dmytrorykun

Differential Revision: D46859120

Pulled By: cipolleschi

fbshipit-source-id: 8878ebaccf6edb801f8e9884e2bf3946380aa748
@cipolleschi cipolleschi force-pushed the cipolleschi/port-download-ci-to-0.72 branch from f73b646 to ec411e5 Compare July 24, 2023 11:54
@cipolleschi
Copy link
Contributor Author

Run the E2E testing with both the new flow (using the token) and the old flow (not using it).

Everything works properly.

Making the PR ready for review. If you want to try the flow out:

  1. checkout this branch locally
  2. wait for CircleCI t finish running (it needs the jobs: test_android, build_and_publish_npm_package-2, build_hermes_macos-Debug)
  3. run the script normally (for the old flow) or passing -c <your circleci token> to test the new one.

@cipolleschi cipolleschi marked this pull request as ready for review July 24, 2023 11:57
@kelset kelset self-requested a review July 24, 2023 12:10
@kelset
Copy link
Collaborator

kelset commented Jul 24, 2023

Test matrix:

old/local flow:
RNTester + iOS + Hermes: ✅ 3m 30s
RNTester + iOS + JSC: ✅ 1m 39s
RNTester + Android + Hermes: ✅ 6m 9s
RNTester + Android + JSC: ✅ 48s
RNTestProject + iOS + Hermes: ✅ (RNTestProject generated in 24m 2s)
RNTestProject + iOS + JSC: ✅ (RNTestProject generated in 24m 2s)
RNTestProject + Android + Hermes: ✅ (RNTestProject generated in 24m 2s)
RNTestProject + Android + JSC: ✅ (RNTestProject generated in 24m 2s)

new/token flow:
RNTester + iOS + Hermes: ✅ 3m 9s
RNTester + iOS + JSC: ✅ 1m 47s
RNTester + Android + Hermes: ✅ 6s
RNTester + Android + JSC: ✅ 6s
RNTestProject + iOS + Hermes: ✅ (RNTestProject generated in 3m 14s)
RNTestProject + iOS + JSC: ✅ (RNTestProject generated in 3m 14s)
RNTestProject + Android + Hermes: ✅ (RNTestProject generated in 3m 14s)
RNTestProject + Android + JSC: ✅ (RNTestProject generated in 3m 14s)

Copy link
Collaborator

@kelset kelset left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested all (see comment above) ✅

:shipit:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner Pick Request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants