ci(regression): speed up the PR gate by dropping the unused input fetch#742
Merged
Conversation
The PR-tier regression gate replays only the cases a pull request touches,
and replay rebuilds the committed bundle from the native output blobs plus
the committed catalog/manifest -- `build_execution_result` never reads the
input datasets. The gate was nonetheless fetching the full CMIP sample data
and every provider's ESGF inputs before replaying, which dominated the job
runtime (tens of minutes) for no benefit.
Remove the `fetch-sample-data` and `test-cases fetch` block from the gate so
it only pays for the small native blobs of the cases it actually replays.
Also make the gate script runnable locally: it emits GitHub Actions log groups
and annotations only under Actions, prints plain output otherwise, and accepts
an optional base ref (defaulting to origin/${GITHUB_BASE_REF:-main}). A
`make regression-gate` target is the convenient entry point.
Update the regression-baselines reference and the testing-diagnostics how-to to
document the no-input-data behaviour and the local entry point.
Codecov Report✅ All modified and coverable lines are covered by tests.
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
lewisjared
added a commit
that referenced
this pull request
Jun 18, 2026
…to feat/regression-ilamb-baselines * origin/ci/faster-regression-pr-gate: docs(changelog): add fragment for PR #742 ci(regression): speed up the PR gate by dropping the unused input fetch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The PR-tier regression baseline gate (
regression-pr-gate.yaml) was taking tens of minutes per run (e.g. this run), almost entirely spent downloading data it never used.For each case a pull request touches, the gate routes to
replay, which rebuilds the committed CMEC bundle from the native output blobs plus the committedcatalog.yaml/manifest.json.build_execution_resultreads only its output directory — it never opens the input datasets.Changes
scripts/ci/regression-pr-gate.shno longer runsfetch-sample-dataortest-cases fetch.::error::/::warning::annotations only under Actions, prints plain output otherwise, and accepts an optional base ref (defaultorigin/${GITHUB_BASE_REF:-main}). A newmake regression-gatetarget is the convenient entry point, so a contributor can reproduce a PR's verdict before pushing.Checklist
Please confirm that this pull request has done the following:
changelog/