You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Gerrit Required Verify workflow in this repository fails routinely, and
each failure emails the change author and the releng/collab-it lists. This is
not a recent regression — it has been steady for at least the last 35 days.
Sample: the last 200 failed runs (2026-06-30 → 2026-08-04), against 200
consecutive runs over 2026-07-27 → 2026-08-04 for the rate.
Measure
Value
Failure rate (last 200 runs)
44 failed, 6 cancelled, 150 succeeded (22%)
Failure notifications
~5.7 per day
Attributable to cause A below
182 of 200 failures (91%)
Attributable to cause B below
16 of 200 failures (8%)
Genuine INFO.yaml validation failures
1 of 200 (working as intended)
Neither dominant cause is a fault in this repository's own content. Both have
upstream fixes, linked below.
Cause A — verify is dispatched for private Gerrit changes (91%)
lfit/releng-gerrit_to_platform dispatches on patchset-created regardless of
whether the change is private. The Gerrit hook runs server-side with full
visibility; the fdio.required.gha account that has to clear and set the vote
does not have it. Both the prepare and vote jobs then die with:
error: fatal: Not found: 46421
fatal: one or more reviews failed; review output above
Uploading private and publishing later is a normal VPP developer workflow — 9 of
the last 60 vpp master changes did it — and every patchset pushed while
private produces a failure email.
Granting the service account visibility of private changes would not fix this:
private change refs are not replicated to GitHub, so the verify job could not
fetch the change anyway.
Importantly, unsetting private fires a fresh dispatch on its own, so suppressing
the earlier dispatches costs no verification coverage.
Cause B — shallow checkout when the change has already merged (8%)
gerrit-compose-required-info-yaml-verify.yaml checks isolation with git diff --name-only HEAD~1 over a fetch-depth: 1 checkout. For CSIT oper-* cherry-picks, which are submitted pre-approved and merge within seconds
of the dispatch, the branch tip is the change commit, so HEAD~1 lies beyond
the shallow boundary:
fatal: ambiguous argument 'HEAD~1': unknown revision or path not in the working tree.
##[error]Process completed with exit code 128.
Worked example: run 30809172607
for change 46399, merged at 11:21:02,
dispatched at 11:21:06.
Independent of the two causes above, both Gerrit workflows here reference
retired sources:
lfit/gerrit-review-action — the action now lives under lfreleng-actions
technote-space/workflow-conclusion-action — archived November 2023, targets
Node.js 20, and is the source of the deprecation warnings that appear as
annotations on every run
lfit/releng-reusable-workflows/...@main — unpinned; behaviour can change
without a corresponding change here
Proposed actions
Short term, here. Add a visibility gate to gerrit-required-verify-non-voting.yaml: query Gerrit as the voting account
before doing anything, and skip the remaining jobs when the change is not
readable. This removes ~91% of the failure emails immediately and is safe to
revert once the dispatcher is fixed. A change implementing this is ready and
can be pushed for review on request.
Short term, here. Refresh the action references above and pin the
reusable workflow call to a release SHA.
Summary
The
Gerrit Required Verifyworkflow in this repository fails routinely, andeach failure emails the change author and the releng/collab-it lists. This is
not a recent regression — it has been steady for at least the last 35 days.
Sample: the last 200 failed runs (2026-06-30 → 2026-08-04), against 200
consecutive runs over 2026-07-27 → 2026-08-04 for the rate.
Neither dominant cause is a fault in this repository's own content. Both have
upstream fixes, linked below.
Cause A — verify is dispatched for private Gerrit changes (91%)
lfit/releng-gerrit_to_platformdispatches onpatchset-createdregardless ofwhether the change is private. The Gerrit hook runs server-side with full
visibility; the
fdio.required.ghaaccount that has to clear and set the votedoes not have it. Both the
prepareandvotejobs then die with:Worked example — change 46421:
Not found: 46421Not found: 46421Uploading private and publishing later is a normal VPP developer workflow — 9 of
the last 60
vppmaster changes did it — and every patchset pushed whileprivate produces a failure email.
Granting the service account visibility of private changes would not fix this:
private change refs are not replicated to GitHub, so the verify job could not
fetch the change anyway.
Importantly, unsetting private fires a fresh dispatch on its own, so suppressing
the earlier dispatches costs no verification coverage.
Upstream fix: lfit/releng-gerrit_to_platform#116
Cause B — shallow checkout when the change has already merged (8%)
gerrit-compose-required-info-yaml-verify.yamlchecks isolation withgit diff --name-only HEAD~1over afetch-depth: 1checkout. For CSIToper-*cherry-picks, which are submitted pre-approved and merge within secondsof the dispatch, the branch tip is the change commit, so
HEAD~1lies beyondthe shallow boundary:
Worked example: run 30809172607
for change 46399, merged at 11:21:02,
dispatched at 11:21:06.
Upstream fixes: lfreleng-actions/checkout-gerrit-change-action#129 and
lfit/releng-reusable-workflows#843
Contributing factors in this repository
Independent of the two causes above, both Gerrit workflows here reference
retired sources:
lfit/gerrit-review-action— the action now lives underlfreleng-actionstechnote-space/workflow-conclusion-action— archived November 2023, targetsNode.js 20, and is the source of the deprecation warnings that appear as
annotations on every run
lfit/releng-reusable-workflows/...@main— unpinned; behaviour can changewithout a corresponding change here
Proposed actions
gerrit-required-verify-non-voting.yaml: query Gerrit as the voting accountbefore doing anything, and skip the remaining jobs when the change is not
readable. This removes ~91% of the failure emails immediately and is safe to
revert once the dispatcher is fixed. A change implementing this is ready and
can be pushed for review on request.
reusable workflow call to a release SHA.
while a change is unreadable — this is the correct fix and benefits every LF
project, not only FD.io.
actively maintained home: Feat: host the Gerrit required-verify reusable workflows lfreleng-actions/generic-workflows#23
Happy to supply the full run classification or raw logs behind any of the
figures above.