Skip to content

Optimize github workflows size/speed/cost #91118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
roryabraham merged 7 commits into from
May 21, 2026
+75 −46
Merged

Optimize github workflows size/speed/cost #91118

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
38f5864
feat: split perf tests into parallel jobs
coleaeason May 19, 2026
5370c81
fix: rightsize runners from blacksmith analysis
coleaeason May 19, 2026
4c880a3
touch file to trigger tests
coleaeason May 19, 2026
f62b97e
feat: use blacksmith checkout caching
coleaeason May 19, 2026
f84036c
fix: no need for deep checkout and manual merge
coleaeason May 19, 2026
b86a129
revert test triggering change
coleaeason May 20, 2026
ab1be28
Fix: actionlint error
coleaeason May 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/buildWeb.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ on:
jobs:
build:
name: Build Web
runs-on: blacksmith-32vcpu-ubuntu-2404
runs-on: blacksmith-8vcpu-ubuntu-2404
outputs:
TAR_FILENAME: webBuild.tar.gz
ZIP_FILENAME: webBuild.zip
Expand Down
117 changes: 73 additions & 44 deletions .github/workflows/reassurePerformanceTests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,62 +7,91 @@ on:
paths-ignore: [docs/**, help/**, .github/**, contributingGuides/**, tests/**, '**.md', '**.sh']

jobs:
perf-tests:
baseline-perf-tests:
if: ${{ github.actor != 'OSBotify' }}
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- name: Checkout baseline branch
# v1
uses: useblacksmith/checkout@41cdeedae8edb2e684ba22896a5fd2a3cb85db6b

- name: Checkout baseline branch
shell: bash
run: |
set -e
BASELINE_BRANCH=${BASELINE_BRANCH:="main"}
git fetch origin "$BASELINE_BRANCH" --no-tags
git checkout --force --detach "origin/$BASELINE_BRANCH" --no-recurse-submodules

- name: Setup NodeJS
uses: ./.github/actions/composite/setupNode

- name: Run baseline performance tests
shell: bash
run: NODE_OPTIONS=--experimental-vm-modules npx reassure --baseline

- name: Upload baseline performance results
# v6
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f
with:
name: reassure-baseline-results
path: ./.reassure/baseline.perf
include-hidden-files: true

branch-perf-tests:
if: ${{ github.actor != 'OSBotify' }}
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- name: Checkout
# v1
uses: useblacksmith/checkout@41cdeedae8edb2e684ba22896a5fd2a3cb85db6b

- name: Setup NodeJS
uses: ./.github/actions/composite/setupNode

- name: Run branch performance tests
shell: bash
env:
BRANCH: ${{ github.head_ref }}
COMMIT_HASH: ${{ github.event.pull_request.head.sha }}
run: NODE_OPTIONS=--experimental-vm-modules npx reassure --branch="$BRANCH" --commit-hash="$COMMIT_HASH" --no-compare

- name: Upload branch performance results
# v6
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f
with:
fetch-depth: 0
name: reassure-branch-results
path: ./.reassure/current.perf
include-hidden-files: true

validate-perf-tests:
if: ${{ github.actor != 'OSBotify' }}
runs-on: blacksmith-4vcpu-ubuntu-2404
needs: [baseline-perf-tests, branch-perf-tests]
Comment on lines +67 to +70
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Run baseline and branch perf tests on the same runner

Splitting measurement into baseline-perf-tests and branch-perf-tests makes baseline and current data come from different runners, which invalidates Reassure comparisons because machine variance becomes part of the delta. Reassure’s own guidance says baseline/current should be measured on the same machine (ideally sequentially), so this change can produce flaky false positives/negatives in validateReassureOutput even when PR code has no real regression.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

@chuckdries chuckdries May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, they use the same type of runner, so they should be about the same, right?

Copy link
Copy Markdown
Contributor

@roryabraham roryabraham May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reassure tests should be deterministic, because they count renders not time. So I think it's fine to run them on separate runners.

steps:
- name: Checkout
# v1
uses: useblacksmith/checkout@41cdeedae8edb2e684ba22896a5fd2a3cb85db6b

- name: Setup NodeJS
uses: ./.github/actions/composite/setupNode

- name: Set dummy git credentials
run: |
git config --global user.email "test@test.com"
git config --global user.name "Test"
- name: Download baseline performance results
# v7
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131
with:
name: reassure-baseline-results
path: ./.reassure

- name: Get common ancestor commit
run: |
git fetch origin main
common_ancestor=$(git merge-base "${{ github.sha }}" origin/main)
echo "COMMIT_HASH=$common_ancestor" >> "$GITHUB_ENV"
- name: Download branch performance results
# v7
uses: actions/download-artifact@37930b1c2abaa49bbe596cd826c3c89aef350131
with:
name: reassure-branch-results
path: ./.reassure

- name: Clean up deleted files
run: |
DELETED_FILES=$(git diff --name-only --diff-filter=D "$COMMIT_HASH" "${{ github.sha }}")
for file in $DELETED_FILES; do
if [ -n "$file" ]; then
rm -f "$file"
echo "Deleted file: $file"
fi
done
- name: Capture PR head SHA
if: ${{ github.event_name == 'pull_request' }}
run: echo "PR_SHA=${{ github.event.pull_request.head.sha }}" >> "$GITHUB_ENV"

- name: Run performance testing script
shell: bash
run: |
set -e
BASELINE_BRANCH=${BASELINE_BRANCH:="main"}
git fetch origin "$BASELINE_BRANCH" --no-tags --depth=1

git checkout --force --detach "origin/$BASELINE_BRANCH" --no-recurse-submodules
npm install --force || (rm -rf node_modules && npm install --force)
NODE_OPTIONS=--experimental-vm-modules npx reassure --baseline

git checkout --force --detach "${PR_SHA:-${GITHUB_SHA}}" --no-recurse-submodules

if ! git merge --no-commit --allow-unrelated-histories "$BASELINE_BRANCH" -X ours; then
git checkout --ours .
fi

npm install --force || (rm -rf node_modules && npm install --force)
NODE_OPTIONS=--experimental-vm-modules npx reassure --branch
- name: Compare performance results
run: node -e "require('@callstack/reassure-compare').compare().catch((error) => { console.error(error); process.exit(1); })"

- name: Validate output.json
id: validateReassureOutput
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ concurrency:
jobs:
jest:
if: ${{ github.event.head_commit.author.name != 'OSBotify' && github.event.head_commit.author.name != 'imgbot[bot]' || github.event_name == 'workflow_call' }}
runs-on: blacksmith-4vcpu-ubuntu-2404
runs-on: blacksmith-8vcpu-ubuntu-2404
env:
CI: true
NODE_OPTIONS: "--experimental-vm-modules --max-old-space-size=8192"
Expand Down
Loading