chore(ci): implement best practices for CI caching #17873
+269
−56
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,6 +33,10 @@ jobs: | |
| matrix: | ||
| working_directory: | ||
| ['tests', 'packages/cloud_firestore/cloud_firestore/example'] | ||
| env: | ||
| AVD_ARCH: x86_64 | ||
| AVD_API_LEVEL: 34 | ||
| AVD_TARGET: google_apis | ||
| steps: | ||
| - uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938 | ||
| - uses: actions/setup-node@1d0ff469b7ec7b3cb9d8673fde0c81c44821de2a | ||
|
|
@@ -43,11 +47,20 @@ jobs: | |
| with: | ||
| distribution: 'temurin' | ||
| java-version: '17' | ||
| - name: 'Install Tools' | ||
| run: | | ||
| sudo npm i -g firebase-tools | ||
| echo "FIREBASE_TOOLS_VERSION=$(npm firebase --version)" >> $GITHUB_ENV | ||
|
Contributor
Author
There was a problem hiding this comment. discussion on how to share information between workflow steps, including syntax for command interpolation while setting an environment variable: https://stackoverflow.com/a/57989070/9910298 example output of command executed so you may verify it is valid for an environment variable (no spaces or similar): |
||
| - name: Firebase Emulator Cache | ||
| id: firebase-emulator-cache | ||
| uses: actions/cache/restore@1bd1e32a3bdc45362d1e726936510720a7c30a57 | ||
|
Contributor
Author
There was a problem hiding this comment. splitting to the separate restore/save actions vs the combo 'cache' action so we may restore every time but only save conditionally |
||
| continue-on-error: true | ||
| with: | ||
| # The firebase emulators are pure javascript and java, OS-independent | ||
| enableCrossOsArchive: true | ||
|
Contributor
Author
There was a problem hiding this comment. documentation if interested https://github.com/actions/cache/blob/main/tips-and-workarounds.md#cross-os-cache complete listing of files in |
||
| # Must match the save path exactly | ||
| path: ~/.cache/firebase/emulators | ||
| key: firebase-emulators-v3-${{ env.FIREBASE_TOOLS_VERSION }} | ||
| restore-keys: firebase-emulators-v3 | ||
| - uses: subosito/flutter-action@f2c4f6686ca8e8d6e6d0f28410eeef506ed66aff | ||
| with: | ||
|
|
@@ -61,9 +74,6 @@ jobs: | |
| melos-version: '5.3.0' | ||
| - name: 'Bootstrap package' | ||
| run: melos bootstrap --scope tests && melos bootstrap --scope "cloud_firestore*" | ||
|
Contributor
Author
There was a problem hiding this comment. Had to move tools install above tools cache restore so it could get the version for cache key |
||
| - name: Start Firebase Emulator | ||
| run: cd ./.github/workflows/scripts && ./start-firebase-emulator.sh | ||
| - name: Enable KVM | ||
|
|
@@ -82,19 +92,21 @@ jobs: | |
| remove-docker-images: true | ||
| remove-large-packages: true | ||
| - name: AVD cache | ||
| uses: actions/cache/restore@1bd1e32a3bdc45362d1e726936510720a7c30a57 | ||
| continue-on-error: true | ||
| id: avd-cache | ||
| with: | ||
| # Must match the save path exactly | ||
| path: | | ||
| ~/.android/avd/* | ||
| ~/.android/adb* | ||
| key: avd-${{ runner.os }}-${{ env.AVD_API_LEVEL }}-${{ env.AVD_TARGET }}-${{ env.AVD_ARCH }} | ||
| - name: Start AVD then run E2E tests | ||
| uses: reactivecircus/android-emulator-runner@b530d96654c385303d652368551fb075bc2f0b6b | ||
|
Contributor
Author
There was a problem hiding this comment. verify visually by noticing the |
||
| with: | ||
| api-level: ${{ env.AVD_API_LEVEL }} | ||
| target: ${{ env.AVD_TARGET }} | ||
| arch: ${{ env.AVD_ARCH }} | ||
| working-directory: ${{ matrix.working_directory }} | ||
| script: | | ||
| flutter test integration_test/e2e_test.dart --ignore-timeouts --dart-define=CI=true -d emulator-5554 | ||
|
|
@@ -103,3 +115,21 @@ jobs: | |
| # https://github.com/ReactiveCircus/android-emulator-runner/issues/385 | ||
| run: | | ||
| pgrep -f appium && pkill -f appium || echo "No Appium process found" | ||
| - name: Save Firestore Emulator Cache | ||
| # Branches can read main cache but main cannot read branch cache. Avoid LRU eviction with main-only cache. | ||
| if: github.ref == 'refs/heads/main' | ||
|
Contributor
Author
There was a problem hiding this comment. example of conditional step based on branch ref https://stackoverflow.com/a/58142412/9910298 |
||
| uses: actions/cache/save@1bd1e32a3bdc45362d1e726936510720a7c30a57 | ||
| with: | ||
| key: ${{ steps.firebase-emulator-cache.outputs.cache-primary-key }} | ||
| # Must match the restore path exactly | ||
| path: ~/.cache/firebase/emulators | ||
| - name: Save Android Emulator Cache | ||
| # Branches can read main cache but main cannot read branch cache. Avoid LRU eviction with main-only cache. | ||
| if: github.ref == 'refs/heads/main' | ||
| uses: actions/cache/save@1bd1e32a3bdc45362d1e726936510720a7c30a57 | ||
| with: | ||
| key: ${{ steps.avd-cache.outputs.cache-primary-key }} | ||
| # Must match the restore path exactly | ||
| path: | | ||
| ~/.android/avd/* | ||
| ~/.android/adb* | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
env var used to follow "Don't Repeat Yourself" programming law since these values will be used both for cache key and emulator action now