Skip to content

fix(ci): pin /vitest and /e2e runs to a single commit, and fix the vitest yarn cache - #5554

Merged
adrians5j merged 2 commits into
nextfrom
claude/pin-pr-sha-and-fix-needs
Aug 4, 2026
Merged

fix(ci): pin /vitest and /e2e runs to a single commit, and fix the vitest yarn cache#5554
adrians5j merged 2 commits into
nextfrom
claude/pin-pr-sha-and-fix-needs

Conversation

@adrians5j

@adrians5j adrians5j commented Aug 4, 2026

Copy link
Copy Markdown
Member

What changed

Two related fixes to the /vitest and /e2e workflows, one commit each.

1. Give the /vitest jobs a direct baseBranch dependency

The vitest-*-constants and vitest-*-run jobs use ${{ needs.baseBranch.outputs.base-branch }} as their checkout path and working directory, but needs only exposes a job's direct dependencies and they reached baseBranch only transitively. The expression resolved to an empty string, so those jobs checked out at the workspace root and every path in them silently lost its prefix.

The most expensive symptom was the yarn cache, configured with the absolute path /.yarn/cache:

Path Validation Error: Path(s) specified in the action for caching do(es) not exist,
hence no cache is being saved

That path never exists, so the cache never saved and those jobs re-downloaded every dependency on every run. It is also why tar -C needed a . fallback in #5550.

Adding baseBranch to their needs makes the expression resolve. Every step in those jobs is already scoped by that same expression — checkout path, yarn cache path, build-cache extraction, install/build, and the test command — so they all move together into the subdirectory that was always intended.

I audited every workflow for this class of bug by checking each job's needs.<job> references against its declared needs. Exactly ten jobs were affected here, plus three in the frozen v5_PullRequestsCommandJest workflow, which is left alone.

2. Pin every job in a run to one commit

Each job ran gh pr checkout <n> independently, and that resolves the PR head at that job's start time. Jobs in one run start tens of minutes apart — on a recent /e2e run the build job checked out at 14:38:04 and a consumer at 15:10:35 — so a push landing mid-run makes the build job produce output from one commit while later jobs run against another.

This was observed for real, not hypothesised. On run 30919777205 (PR #5549):

Time Event
14:33:51 9b71fa2e6 — PR head, no cli-core changes
14:38:04 build checks out → gets 9b71fa2e6
14:50:20 4306b831c lands, changing packages/cli-core/files/references.json
15:10:35 consumer checks out → gets 280bdfa55, including that change

getPackageSourceHash hashes the whole package folder excluding only dist, lib, node_modules and tsconfig.build.tsbuildinfo, so files/references.json counts. The consumer computed a different hash for @webiny/cli-core than the build job had recorded and correctly rebuilt it.

The wasted 2.5s is irrelevant. Two jobs in one run disagreeing about what they are testing is not: a green run stops meaning "this commit passed" and a red one is hard to reproduce.

baseBranch now resolves the head SHA once, exposes it as pr-sha, and every checkout detaches onto it:

gh pr checkout ${{ github.event.issue.number }}
git checkout --detach ${{ needs.baseBranch.outputs.pr-sha }}

Verified that all 11 pinned checkout sites across the two workflows have baseBranch as a direct need, so pr-sha actually resolves in each.

Not included

  • /alpha and /beta have the same drift, via ref: ${{ needs.prBranch.outputs.pr-branch }} on their checkout steps — a branch name, re-resolved per job. They use actions/checkout rather than gh pr checkout, so the fix is shaped differently and belongs in its own PR.
  • The vitest-*-constants jobs check out the base branch, not the PR. They have no Checkout Pull Request step, so listVitestTestCommands discovers test commands from next rather than from the PR — meaning tests added by a PR may never be scheduled. Pre-existing and worth its own investigation.

Changelog

More reliable test and release commands

Test runs triggered from a pull request could end up building one version of the code and testing another if the branch was updated while the run was in progress; each run is now pinned to a single commit. Test jobs also cache their dependencies correctly again instead of re-downloading them every time.

Squash Merge Commit

fix(ci): pin /vitest and /e2e runs to one commit, fix yarn cache (#5554)

adrians5j and others added 2 commits August 4, 2026 17:43
The `vitest-*-constants` and `vitest-*-run` jobs reference
`${{ needs.baseBranch.outputs.base-branch }}` as their checkout path and working
directory, but `needs` only exposes a job's DIRECT dependencies and they reached
`baseBranch` only transitively. The expression resolved to an empty string, so
those jobs checked the repository out at the workspace root and every path in
them silently lost its prefix.

Most visibly the yarn cache was configured with the absolute path
`/.yarn/cache`, which never exists - hence the long-standing warning

  Path Validation Error: Path(s) specified in the action for caching
  do(es) not exist, hence no cache is being saved

meaning those jobs re-downloaded every dependency on every run. It is also why
`tar -C` had to be given a `.` fallback in #5550.

Adding `baseBranch` to their `needs` makes the expression resolve. Every step in
those jobs is already scoped by the same expression (checkout path, yarn cache
path, build cache extraction, install/build, test command), so they all move
together into the subdirectory that was always intended.

Audited every workflow for the same class of bug: only these ten jobs and three
jobs in the frozen v5 Jest workflow were affected. The frozen file is left alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each job ran `gh pr checkout <n>` independently, which resolves the PR head at
that job's start time. Jobs in a single run start tens of minutes apart - on a
recent /e2e run the `build` job checked out at 14:38 and a consumer at 15:10 -
so a push landing mid-run made the build job produce output from one commit
while later jobs ran against another.

That was observed for real: a commit at 14:50 changed
`packages/cli-core/files/references.json`, so the consumer computed a different
source hash for `@webiny/cli-core` than the build job had recorded and correctly
rebuilt it. The wasted 2.5s does not matter; two jobs in one run disagreeing
about what they are testing does.

`baseBranch` now resolves the head SHA once and exposes it as `pr-sha`, and
every checkout detaches onto it. A run is pinned to a single commit, so a green
result means one commit passed rather than some mixture, and a red one is
reproducible.

/alpha and /beta have the same exposure through `ref: <branch name>` on their
checkout steps, but they check out the PR branch rather than using
`gh pr checkout`, so they are left for a separate change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@adrians5j
adrians5j force-pushed the claude/pin-pr-sha-and-fix-needs branch from beeff43 to fef394f Compare August 4, 2026 15:45
@adrians5j
adrians5j merged commit 0d717d1 into next Aug 4, 2026
12 of 13 checks passed
@adrians5j
adrians5j deleted the claude/pin-pr-sha-and-fix-needs branch August 4, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant