fix(gate-4): audit the LOCK, and stop reporting "could not audit" as "CVEs" - #135
Merged
Conversation
…build outputs
`head` closes the pipe as soon as it has its 40 lines, `tr` dies of SIGPIPE, and
`set -o pipefail` three lines above turns that into a FAILED JOB — for a
log-preview line with no bearing on whether the build worked.
tr '\0' '\n' < /tmp/frontend-build-files.z | head -40
It only fires when the producer is still writing as head leaves, so it is
invisible on small builds and certain on large ones:
openregister build writes <40 files -> Frontend Build passed
nextcloud-vue dist is thousands -> failed EVERY run since the gate
landed (835918f, today 16:40)
Reproduced outside CI. 200k NUL-separated entries:
$ bash -c "set -euo pipefail; tr '\0' '\n' < big.z | head -40 >/dev/null; \
echo REACHED-AFTER"
exit=141 # 128+13 = SIGPIPE, and REACHED-AFTER never prints
$ bash -c "set -euo pipefail; tr '\0' '\n' < big.z | sed -n '1,40p' >/dev/null; \
echo REACHED-AFTER"
REACHED-AFTER
exit=0
`sed` reads its input to the end, so there is nothing to break, and it still
prints exactly 40 lines. The "…" trailer now says how many files were elided
instead of implying there might be none.
SAME BUG, WORSE FAILURE MODE, same file — the Quality Report step:
COVERED=$(grep -oP 'coveredstatements="\K[0-9]+' clover.xml | head -1 || echo "0")
That step also sets pipefail, and here `|| echo "0"` does not rescue the value:
it APPENDS "0" to what head already printed, so the arithmetic below receives
"1234\n0". Measured on a 60k-file clover.xml:
old: COVERED=[0
0]
new: COVERED=[0]
Reporting a wrong coverage number is worse than failing, because nobody
re-verifies a number that rendered. Switched to `grep -oPm1`, which stops grep
itself and removes the pipe rather than working around it.
No other `| head` under pipefail remains in this workflow.
…"CVEs"
Found by the gates' first real CI run, on openbuild#104: gate-4 reported
`composer-audit: FAIL — CVEs or advisories`. There were no CVEs. Auditing the
same lock reports "No security vulnerability advisories found."
`composer audit` with no vendor/ present does not audit the lock, and what it
does instead depends on the composer version. BOTH behaviours are wrong:
composer >= 2.8 "No installed packages found. Please run composer install
... or pass --locked", exit 1 — so the gate announced a
security finding for a run that found nothing and audited
nothing. A configuration error wearing a CVE's clothes.
composer 2.7.x "No packages - skipping audit", exit 0 — a SILENT FAIL-OPEN.
The gate PASSED having audited nothing at all.
The second is the more dangerous one, and it is the reason this could not be
left as "just make CI run composer install first": on any runner with an older
composer, gate-4 has been passing without auditing anything, and that is
indistinguishable from a clean audit in the output.
Now:
- audit `--locked` whenever a composer.lock exists. The lock is the right
object anyway — it is what CI installs and what pins the transitive tree —
and it needs no vendor/, so the gate no longer depends on whether some
earlier step happened to run `composer install`.
- an exit 0 whose output says "no packages" is treated as a FAILURE, not a
pass. Auditing nothing is never a clean audit.
- "could not run" is reported in those words and explicitly NOT as a CVE
finding, so the two are distinguishable at a glance.
Positive control, three directions, in a clean php:8.3-cli container with
composer 2.10.2 and no vendor/:
openbuild's real lock -> [gate-4] PASS ("No security vulnerability
advisories found.")
lock with guzzlehttp/guzzle -> [gate-4] FAIL — CVEs or advisories, and the
6.5.0 pinned in log names the package and the advisories
("Found 16 security vulnerability
advisories affecting 2 packages",
GHSA-h95v-h523-3mw8, ...)
no lock and no vendor -> [gate-4] FAIL — "audit COULD NOT RUN ...
NOT a CVE finding"
Worth noting for the second case: composer 2.10 refuses to resolve a
vulnerable package at all by default (policy.advisories.block), so the control
needed that policy disabled to even produce the lock. The first attempt at it
silently produced no lock — and the new code correctly reported "could not
run" rather than inventing a finding, which is itself the behaviour under test.
rubenvdlinde
added a commit
to ConductionNL/openbuild
that referenced
this pull request
Aug 3, 2026
The first real CI run of the gates on this PR failed on gate-4: `composer-audit: FAIL — CVEs or advisories`. There are no CVEs — auditing this repo's lock properly reports "No security vulnerability advisories found." `composer audit` with no vendor/ present does not audit the lock. On the runner's composer it errors and exits non-zero, which gate-4 reported as a security finding. On composer 2.7.x the same situation prints "No packages - skipping audit" and exits 0 — a silent fail-open that passes having audited nothing. Fixed in the package (ConductionNL/.github#135, released as v1.0.1): gate-4 audits `--locked`, treats an exit 0 that audited no packages as a failure, and reports "could not run" in those words rather than as a CVE finding. Both the workflow pin and composer.lock move to v1.0.1 so the two paths into the gates cannot disagree about which version this repo is on.
rubenvdlinde
added a commit
to ConductionNL/openbuild
that referenced
this pull request
Aug 3, 2026
…104) * build(quality): run the Hydra gates as part of composer check:strict openbuild is the pilot for conduction/hydra-gates. The gates have until now only run inside hydra's own containers, which means an agent finishing a task here could report done without any of the 61 mechanical gates having looked at its diff. Wiring them into check:strict makes "gates pass" part of the definition of done in the same command everything else already runs. The gates are diff-scoped per ADR-020, so this does NOT import openbuild's inherited debt. A full-repo run of this tree fails 16 gates today (img-alt, button-name, table-headers, spec-anchor-existence and friends); scoped to a PR's own diff, only what that PR touched is enforced. `composer gates:full` is available for the audit view, and is deliberately not what check:strict runs. check:strict keeps its 0/1 contract, but the gate exit code is preserved separately and reported, because it carries the failure COUNT and flows route on it. 99 is distinguished from a gate failure in the summary: it means the gates could not run at all, which is a configuration error and not a clean tree. The dependency is declared against a `path` repository at ../hydra, which is the fleet's checkout layout. That is the part of this change that is provisional — see the PR description; how hydra-gates should be DISTRIBUTED (private Packagist, a Satis mirror, or a split public repo) is an open decision, and a path repository is the shape that is verifiable today without one. * build(quality): lock conduction/hydra-gates at dev stability A path repository derives its version from the checked-out branch, so the default minimum-stability of 'stable' rejects a '*' constraint outright: 'found conduction/hydra-gates[dev-main, ...] but it does not match your minimum-stability'. '@dev' is the constraint that resolves, and it keeps working once the package is eventually tagged. * build(quality): resolve hydra-gates from the public package, not a path repo The `path` repository at `../hydra` is what blocked this PR. It works on a developer machine and fails in CI with `Source path "../hydra" is not found`, because the fleet's sibling-checkout layout does not exist in a CI job and hydra is private, so no runner can fetch it. conduction/hydra-gates now lives in ConductionNL/.github, which is PUBLIC and already owns the shared workflows (ConductionNL/.github#131). Pointing at it needs no credentials in this repo or on any runner. - repositories: `path ../hydra` -> `vcs https://github.com/ConductionNL/.github.git` with `"no-api": true`, so composer clones over git instead of the GitHub API and a rate-limited unauthenticated runner cannot become a failed install. - require-dev: `@dev` -> `^1.0`, resolving to the v1.0.0 tag. That also drops the dev-stability requirement the previous commit needed, because a path repository derives its version from the checked-out branch and a tag does not. - composer.lock pins commit fdad2546f2ac68aa64be5fecef898784c1847538. Nothing else changes. `composer gates`, `composer gates:full` and the gate handling inside `check:strict` are untouched, including the part that captures the gate exit code separately: it carries the FAILURE COUNT, and 99 ("could not run at all") is still reported distinctly from a gate failure so a configuration error can never read as a clean tree. Also merged origin/development in. The branch predated the fleet-wide timeout-minutes work, so it was carrying a silent revert of the bounds on exporter-e2e.yml and pull-request-lint-check.yaml. * ci(quality): actually run the gates in CI, not just in a local composer script Wiring the gates into `composer check:strict` was a local-only change. Nothing in this repository's CI invokes check:strict — no workflow does, verified by grepping .github/workflows — so this PR would have merged green with the 61 gates never having executed on any diff. A gate that runs only on a developer's machine is not a gate, and a green that never ran it is the failure mode the gates exist to catch. The shared quality workflow now ships a `hydra-gates` job (ConductionNL/.github#131), opt-in per repo and defaulting to off. openbuild is the pilot, so it opts in. `hydra-gates-ref` is pinned to the v1.0.0 tag rather than tracking `main`, so a change to the gates cannot move this repo's verdict without a commit here to move the pin. The job resolves the diff base from the PR's real target branch, checks out the app with fetch-depth: 0 (a shallow clone has no base to diff against), and reports exit 99 — "could not run at all" — distinctly from "N gates failed". * build(quality): bump hydra-gates to v1.0.1 for the gate-4 audit fix The first real CI run of the gates on this PR failed on gate-4: `composer-audit: FAIL — CVEs or advisories`. There are no CVEs — auditing this repo's lock properly reports "No security vulnerability advisories found." `composer audit` with no vendor/ present does not audit the lock. On the runner's composer it errors and exits non-zero, which gate-4 reported as a security finding. On composer 2.7.x the same situation prints "No packages - skipping audit" and exits 0 — a silent fail-open that passes having audited nothing. Fixed in the package (ConductionNL/.github#135, released as v1.0.1): gate-4 audits `--locked`, treats an exit 0 that audited no packages as a failure, and reports "could not run" in those words rather than as a CVE finding. Both the workflow pin and composer.lock move to v1.0.1 so the two paths into the gates cannot disagree about which version this repo is on. --------- Co-authored-by: Ruben van der Linde <juan.claude@conduction.nl>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by the gates' first real CI run, on ConductionNL/openbuild#104. gate-4 reported
composer-audit: FAIL — CVEs or advisories. There are no CVEs — auditing the same lock reportsNo security vulnerability advisories found.What was actually wrong
composer auditwith novendor/present does not audit the lock, and what it does instead depends on the composer version. Both behaviours are wrong:vendor/>= 2.8No installed packages found. Please run "composer install" ... or pass "--locked", exit 12.7.xNo packages - skipping audit, exit 0The second is the more dangerous one, and it is why this could not be fixed by "just run
composer installin CI first": on any runner with an older composer, gate-4 has been passing without auditing anything, and that output is indistinguishable from a clean audit.The fix
--lockedwhenever acomposer.lockexists. The lock is the right object anyway — it is what CI installs and what pins the transitive tree — and it needs novendor/, so the gate stops depending on whether some earlier step happened to runcomposer install.Positive control — three directions
Clean
php:8.3-clicontainer, composer 2.10.2, novendor/:[gate-4] composer-audit: PASS— log:No security vulnerability advisories found.guzzlehttp/guzzle 6.5.0[gate-4] composer-audit: FAIL — CVEs or advisories, log names the package and advisories:Found 16 security vulnerability advisories affecting 2 packages,GHSA-h95v-h523-3mw8[gate-4] composer-audit: FAIL — audit COULD NOT RUN (no installed packages and no lock to audit) — NOT a CVE findingWorth recording: composer 2.10 refuses to resolve a vulnerable package at all by default (
policy.advisories.block), so the middle control needed that policy disabled to even produce a lock. The first attempt silently produced no lock — and the new code correctly reported "could not run" instead of inventing a finding, which is the behaviour under test.The package's 13 invariant assertions still pass.
Why this matters beyond gate-4
This is the same defect class the package's coverage block exists for: a check that cannot run must never be indistinguishable from a check that ran and found nothing. gate-4 had both failure modes at once depending on the runner, and neither was visible in the output.