fix(ci): pin /vitest and /e2e runs to a single commit, and fix the vitest yarn cache - #5554
Merged
Conversation
The `vitest-*-constants` and `vitest-*-run` jobs reference
`${{ needs.baseBranch.outputs.base-branch }}` as their checkout path and working
directory, but `needs` only exposes a job's DIRECT dependencies and they reached
`baseBranch` only transitively. The expression resolved to an empty string, so
those jobs checked the repository out at the workspace root and every path in
them silently lost its prefix.
Most visibly the yarn cache was configured with the absolute path
`/.yarn/cache`, which never exists - hence the long-standing warning
Path Validation Error: Path(s) specified in the action for caching
do(es) not exist, hence no cache is being saved
meaning those jobs re-downloaded every dependency on every run. It is also why
`tar -C` had to be given a `.` fallback in #5550.
Adding `baseBranch` to their `needs` makes the expression resolve. Every step in
those jobs is already scoped by the same expression (checkout path, yarn cache
path, build cache extraction, install/build, test command), so they all move
together into the subdirectory that was always intended.
Audited every workflow for the same class of bug: only these ten jobs and three
jobs in the frozen v5 Jest workflow were affected. The frozen file is left alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each job ran `gh pr checkout <n>` independently, which resolves the PR head at that job's start time. Jobs in a single run start tens of minutes apart - on a recent /e2e run the `build` job checked out at 14:38 and a consumer at 15:10 - so a push landing mid-run made the build job produce output from one commit while later jobs ran against another. That was observed for real: a commit at 14:50 changed `packages/cli-core/files/references.json`, so the consumer computed a different source hash for `@webiny/cli-core` than the build job had recorded and correctly rebuilt it. The wasted 2.5s does not matter; two jobs in one run disagreeing about what they are testing does. `baseBranch` now resolves the head SHA once and exposes it as `pr-sha`, and every checkout detaches onto it. A run is pinned to a single commit, so a green result means one commit passed rather than some mixture, and a red one is reproducible. /alpha and /beta have the same exposure through `ref: <branch name>` on their checkout steps, but they check out the PR branch rather than using `gh pr checkout`, so they are left for a separate change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
adrians5j
force-pushed
the
claude/pin-pr-sha-and-fix-needs
branch
from
August 4, 2026 15:45
beeff43 to
fef394f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Two related fixes to the
/vitestand/e2eworkflows, one commit each.1. Give the
/vitestjobs a directbaseBranchdependencyThe
vitest-*-constantsandvitest-*-runjobs use${{ needs.baseBranch.outputs.base-branch }}as their checkout path and working directory, butneedsonly exposes a job's direct dependencies and they reachedbaseBranchonly transitively. The expression resolved to an empty string, so those jobs checked out at the workspace root and every path in them silently lost its prefix.The most expensive symptom was the yarn cache, configured with the absolute path
/.yarn/cache:That path never exists, so the cache never saved and those jobs re-downloaded every dependency on every run. It is also why
tar -Cneeded a.fallback in #5550.Adding
baseBranchto theirneedsmakes the expression resolve. Every step in those jobs is already scoped by that same expression — checkout path, yarn cache path, build-cache extraction, install/build, and the test command — so they all move together into the subdirectory that was always intended.I audited every workflow for this class of bug by checking each job's
needs.<job>references against its declaredneeds. Exactly ten jobs were affected here, plus three in the frozenv5_PullRequestsCommandJestworkflow, which is left alone.2. Pin every job in a run to one commit
Each job ran
gh pr checkout <n>independently, and that resolves the PR head at that job's start time. Jobs in one run start tens of minutes apart — on a recent/e2erun thebuildjob checked out at 14:38:04 and a consumer at 15:10:35 — so a push landing mid-run makes the build job produce output from one commit while later jobs run against another.This was observed for real, not hypothesised. On run
30919777205(PR #5549):9b71fa2e6— PR head, nocli-corechangesbuildchecks out → gets9b71fa2e64306b831clands, changingpackages/cli-core/files/references.json280bdfa55, including that changegetPackageSourceHashhashes the whole package folder excluding onlydist,lib,node_modulesandtsconfig.build.tsbuildinfo, sofiles/references.jsoncounts. The consumer computed a different hash for@webiny/cli-corethan the build job had recorded and correctly rebuilt it.The wasted 2.5s is irrelevant. Two jobs in one run disagreeing about what they are testing is not: a green run stops meaning "this commit passed" and a red one is hard to reproduce.
baseBranchnow resolves the head SHA once, exposes it aspr-sha, and every checkout detaches onto it:Verified that all 11 pinned checkout sites across the two workflows have
baseBranchas a direct need, sopr-shaactually resolves in each.Not included
/alphaand/betahave the same drift, viaref: ${{ needs.prBranch.outputs.pr-branch }}on their checkout steps — a branch name, re-resolved per job. They useactions/checkoutrather thangh pr checkout, so the fix is shaped differently and belongs in its own PR.vitest-*-constantsjobs check out the base branch, not the PR. They have noCheckout Pull Requeststep, solistVitestTestCommandsdiscovers test commands fromnextrather than from the PR — meaning tests added by a PR may never be scheduled. Pre-existing and worth its own investigation.Changelog
More reliable test and release commands
Test runs triggered from a pull request could end up building one version of the code and testing another if the branch was updated while the run was in progress; each run is now pinned to a single commit. Test jobs also cache their dependencies correctly again instead of re-downloading them every time.
Squash Merge Commit