You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
codecov/patch posts a red check-run verdict while validate-tests is still running, computed
from a partial coverage report. It later self-corrects, but the red status is real and visible in the
meantime.
after_n_builds: 1 fires the verdict on the first upload to land. The uploads are not ordered by
importance:
upload
flag
job
review-enrichment
rees
validate-code
control-plane
control-plane
validate-code
engine
engine
validate-code
root vitest
backend
validate-tests
validate-code finishes in ~1-2 minutes; validate-tests takes far longer. So the verdict is
routinely computed from rees + engine alone — without the backend lcov that covers most of src/**.
Observed on #9799: codecov/patch went pass (after the rees upload), then fail (after the engine upload), while validate-tests was still pending the whole time.
Why it matters
This is not cosmetic. Per the gate's disposition matrix, any failed CI check is a CLOSE for a
contributor PR — codecov/patch explicitly included. A PR can therefore be auto-closed on a red
status computed from an incomplete report, one that would have resolved green minutes later. The
contributor gets closed for a defect that does not exist, which is the same class of failure as #9798: a CI signal that corresponds to no real problem.
require_ci_to_pass: true gates the final verdict on the CI run's conclusion but does not stop the
interim status from being posted.
Why bumping after_n_builds is not the fix
The existing comment in codecov.yml already rules it out, and it is right:
It must stay 1, not 2 [...] review-enrichment / control-plane add their own flag uploads on PRs
that touch them -- those uploads only exist on such PRs, and a floor above the guaranteed minimum
would leave codecov/patch permanently un-posted on every other PR.
A count cannot express "wait for the backend upload specifically." Any floor high enough to
guarantee backend has landed would hang the check on PRs that produce fewer uploads.
Expected
Use Codecov's explicit completion signal instead of a heuristic count — codecov.notify.manual_trigger
(or the equivalent upload-then-finalize flow), with the finalize step running after every upload for
the commit has been sent. That makes the verdict fire exactly once, on a complete report, regardless
of how many flags a given PR happens to produce.
Whatever the mechanism, the acceptance criterion is: codecov/patch must not post a conclusion
until every coverage upload for that commit has landed.
Notes
Found while validating fix(ci): normalize committed CRLF to LF and name the failure for what it is #9799 (line-ending normalization). That PR is unaffected in the end — its
patch coverage is genuinely green — but the premature red sent the investigation down a false path
for a while, which is exactly the cost this issue describes.
The same premature-window comment in codecov.yml acknowledges the behavior and calls it
self-correcting. It does self-correct, but "transiently red" and "auto-closed" are not compatible
states for a repo whose gate closes on red CI.
Problem
codecov/patchposts a red check-run verdict whilevalidate-testsis still running, computedfrom a partial coverage report. It later self-corrects, but the red status is real and visible in the
meantime.
codecov.ymlsets:after_n_builds: 1fires the verdict on the first upload to land. The uploads are not ordered byimportance:
reesvalidate-codecontrol-planevalidate-codeenginevalidate-codebackendvalidate-testsvalidate-codefinishes in ~1-2 minutes;validate-teststakes far longer. So the verdict isroutinely computed from
rees+enginealone — without thebackendlcov that covers most ofsrc/**.Observed on #9799:
codecov/patchwentpass(after thereesupload), thenfail(after theengineupload), whilevalidate-testswas stillpendingthe whole time.Why it matters
This is not cosmetic. Per the gate's disposition matrix, any failed CI check is a CLOSE for a
contributor PR —
codecov/patchexplicitly included. A PR can therefore be auto-closed on a redstatus computed from an incomplete report, one that would have resolved green minutes later. The
contributor gets closed for a defect that does not exist, which is the same class of failure as
#9798: a CI signal that corresponds to no real problem.
require_ci_to_pass: truegates the final verdict on the CI run's conclusion but does not stop theinterim status from being posted.
Why bumping
after_n_buildsis not the fixThe existing comment in
codecov.ymlalready rules it out, and it is right:A count cannot express "wait for the
backendupload specifically." Any floor high enough toguarantee
backendhas landed would hang the check on PRs that produce fewer uploads.Expected
Use Codecov's explicit completion signal instead of a heuristic count —
codecov.notify.manual_trigger(or the equivalent upload-then-finalize flow), with the finalize step running after every upload for
the commit has been sent. That makes the verdict fire exactly once, on a complete report, regardless
of how many flags a given PR happens to produce.
Whatever the mechanism, the acceptance criterion is:
codecov/patchmust not post a conclusionuntil every coverage upload for that commit has landed.
Notes
patch coverage is genuinely green — but the premature red sent the investigation down a false path
for a while, which is exactly the cost this issue describes.
codecov.ymlacknowledges the behavior and calls itself-correcting. It does self-correct, but "transiently red" and "auto-closed" are not compatible
states for a repo whose gate closes on red CI.