ci: skip heavy jobs on artifacts/docs-only diffs - #353
Merged
Conversation
Every PR — including one that edits a single line of rivet YAML — runs Kani, Miri, mutation testing, fuzz smoke, Bazel, coverage and a Mathlib lake build. That is hours of self-hosted runner time buying nothing: a diff confined to artifacts/*.yaml, safety/, docs/ or markdown cannot change a proof, a mutant, a fuzz target or a Miri run. Add a `changes` job to ci.yml and proofs.yml that classifies the PR diff once, and gate the 13 expensive jobs on its output. The jobs are SKIPPED, never removed. A job-level `if:` reports a `skipped` conclusion, which *satisfies* a required status check, so the 14-context branch-protection gate stays non-empty and intact. Filtering at the workflow level (`on: paths:`) would instead leave those required checks permanently *pending* and block every merge — the comment in ci.yml says so, because the distinction is easy to "simplify" away. The filter is fail-safe in the code direction: an allow-list names the non-code paths and anything outside it counts as code, so a new top-level path runs the full suite until deliberately allow-listed. A mixed diff (artifacts + Rust) is classified as code — the count is over non-matching files, not a `grep -q` on matching ones, so the decision is both explicit and readable in the job log. Non-PR events (push to main, tags, schedules) always run everything. fmt, audit, deny and rivet-validate stay ungated: they are cheap, and rivet-validate is precisely the check an artifacts-only PR must pass. Verified against 12 hand-built diffs (artifacts-only, multi-artifact, root/nested markdown, docs/, safety/, Rust, mixed, workflow, manifest, Lean, non-yaml artifact): all 12 classify as intended. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rivet verification gate✅ 20/20 passed
Filter: Failed artifacts(none) Updated automatically by |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every PR — including one that edits a single line of rivet YAML — runs Kani, Miri, mutation testing, fuzz smoke, Bazel, coverage and a Mathlib
lake build. That is hours of self-hosted runner time buying nothing: a diff confined toartifacts/*.yaml,safety/,docs/or markdown cannot change a proof, a mutant, a fuzz target or a Miri run.What
A
changesjob inci.ymlandproofs.ymlclassifies the PR diff once; 13 expensive jobs consume its output.code == 'true'fmt/audit/deny/rivet-validatestay ungated: they are cheap, andrivet-validateis precisely the check an artifacts-only PR must pass.The load-bearing detail
The jobs are skipped, never removed. A job-level
if:reports askippedconclusion, which satisfies a required status check, so the 14-context branch-protection gate stays non-empty and intact. Filtering at the workflow level (on: paths:) would instead leave those required checks permanently pending and block every merge. The ci.yml comment states this explicitly, because the distinction is exactly the kind of thing a later "simplification" removes.Fail-safe direction is toward code: an allow-list names the non-code paths and anything outside it counts as code, so a new top-level path runs the full suite until it is deliberately allow-listed. A mixed diff (artifacts and Rust) classifies as code — the gate counts non-matching files rather than
grep -q-ing matching ones, so the decision is both explicit and readable in the job log. Non-PR events (push to main, tags, schedules) always run everything.Verification
The filter was exercised against 12 hand-built diffs in
bash(CI's shell) — artifacts-only, multi-artifact, root markdown, nested markdown,docs/,safety/, Rust, mixed, workflow, manifest, Lean, non-yaml artifact. All 12 classify as intended.This PR touches
.github/**, so it classifies as code and runs the full 14-check suite — which is the correct self-test of the gate. The next artifacts-only PR is what empirically confirms the skip path.Follow-up required after merge (do not skip)
If the
changesjob itself fails (runner error, checkout failure), GitHub skips its dependents for dependency-failure — also reported asskipped, which would satisfy all required checks with zero tests run. Closing that hole means addingDetect changed pathsandDetect changed paths (proofs)to the required contexts (14 → 16). That must happen after this merges, since a required context that no run reports would leave existing PRs pending; open PRs then need a branch update.🤖 Generated with Claude Code