fix(quality): two dead gates in the coverage ratchet — an artifact that never existed, and a job deleted by an unrelated failure - #156
Open
rubenvdlinde wants to merge 2 commits into
Conversation
…er uploaded update-baseline tells you to close the gate by committing 'the recomputed value attached as the coverage-baseline artifact of this run'. That artifact has never existed: .coverage-baseline is a dotfile and actions/upload-artifact@v4 excludes hidden files by default, then warns rather than fails when the glob matches nothing. The gate asked you to close it with a file it had silently declined to produce. Reproduced on docudesk run 30925609908, where the preceding git diff prints the changed file three lines before the upload reports it missing.
update-baseline's if carried no status function, so GitHub wrapped it in an implicit success() over the whole needs closure. phpunit survives a red php-quality because its own if has !cancelled(); update-baseline had no such escape, so a failing PHPMD - a grandparent it does not depend on - removed the coverage ratchet while phpunit itself was green. Measured on procest: five consecutive push runs on development where php-quality failed, phpunit succeeded, and the ratchet reported nothing. Reproduced twice on doriath today. Adds !cancelled() && needs.phpunit.result == 'success', mirroring phpunit.
This was referenced Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
update-baseline("Coverage Baseline Check") hard-fails when measured coveragehas drifted past the committed baseline, and its error tells you how to close
the gate:
That artifact has never existed.
.coverage-baselineis a dotfile, andactions/upload-artifact@v4excludes hidden files by default(
include-hidden-files: false). When the glob then matches nothing the actiondoes not fail — it prints a warning and the step still succeeds:
So the gate instructs you to close it using a file it has just silently declined
to produce. That is an unclosable gate, and it is the same failure shape this
job was previously rewritten to remove (the swallowed
git pushthat made arejected push read as work having been done).
Reproduced, not inferred
docudesk, run
30925609908,
the first ever execution of this job in that repo. In one job log, in order:
Measured coverage has moved beyond the committed baseline:git --no-pager diff -- .coverage-baseline→ prints the diff, proving thefile is on disk and modified.
##[warning]No files were found with the provided path: .coverage-baseline.Step 2 and step 3 are three lines apart and contradict each other. Only the
hidden-file default explains both.
The fix
include-hidden-files: trueon that one upload step, with the reasoning recordedinline so the next person does not have to re-derive it.
Not fixed here, but found while confirming it — please read
Push runs are being cancelled by concurrent
pull_requestruns on merge intodevelopment, which voids every push-only gate. Measured across the 22 reposwhose callers I touched today, 5 had their post-merge push run
cancelledwithin seconds:
The job cap is 45 minutes, so none of these is a timeout — they are
concurrency: cancel-in-progresskills. And it is not new: hermiq's 14:31 pushrun and nldesign's 12:55 push run were cancelled the same way, before any change
of mine.
This matters more than it looks, because
SBOM,Features ExtractandCoverage Baseline Checkonly ever run onpush. When the push run dies, thosethree reach no verdict at all — and a cancelled run is not a red, so nothing
announces it. It is the same "absence looks like success" problem the opt-in
gates had, one level up.
I have not changed the
concurrencyblock here: it is fleet-wide, severalcallers override it with their own group expression, and getting it wrong would
be worse than the current state. Flagging it with the measurements rather than
guessing at a fix.
Second bug, found while confirming the first: a failing PHPMD deletes the coverage ratchet
update-baseline's condition carried no status function:A job condition with no status function is implicitly wrapped in
success(),and that
success()is not satisfied by the declared dependency alone — afailure anywhere in the
needsclosure removes the job.phpunitsits behindneeds: [php-quality, security]and survives a redphp-qualityonlybecause its own
ifcarries!cancelled(). This job had no such escape, so afailing PHPMD — a grandparent it does not depend on — silently deleted the
coverage ratchet while
phpunititself was green.Measured, on procest's own push history
Five consecutive runs in which the ratchet reported nothing, in a repo that has
had
enable-coverage-guard: trueall along — and nothing announced it, becausea skipped job is not a red. Reproduced on doriath twice today (run
30925445843 and its re-run):
steps: [],started_at == completed_at, the shapeof a job that never existed.
Fix
Add
!cancelled() && needs.phpunit.result == 'success', mirroring whatphpunitalready does one level up. It also makes the intent legible: this job needs a
successful phpunit because it downloads that job's coverage artifact. It has
no opinion about PHPMD.
Why this matters beyond one job
Both bugs in this PR have the same signature as the ones #147/#148/#149/#150
addressed: the check's absence is indistinguishable from its success. One
produced an artifact that never existed; the other removed a job entirely. In
neither case did anything go red.