Skip to content

build(quality): run the Hydra gates as part of composer check:strict - #104

Merged
rubenvdlinde merged 6 commits into
developmentfrom
feat/wire-hydra-gates-check-strict
Aug 3, 2026
Merged

build(quality): run the Hydra gates as part of composer check:strict#104
rubenvdlinde merged 6 commits into
developmentfrom
feat/wire-hydra-gates-check-strict

Conversation

@rubenvdlinde

@rubenvdlinde rubenvdlinde commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Why

openbuild is the pilot for conduction/hydra-gates (ConductionNL/hydra#504). The gates have until now only run inside hydra's containers, so an agent finishing a task here could report done without any of the 61 mechanical gates having looked at its diff. Wiring them into check:strict makes "gates pass" part of the definition of done in the command everything else already runs.

Scoping — this does not import inherited debt

The gates are diff-scoped per ADR-020. A full-repo run of this tree fails 16 gates today (img-alt, button-name, table-headers, spec-anchor-existence, semantic-controls, window-confirm, e2e-networkidle and friends). Scoped to a PR's own diff, only what that PR touched is enforced, and this branch is green.

composer gates:full gives the audit view and is deliberately not what check:strict runs.

Exit codes

check:strict keeps its 0/1 contract, but the gate exit code is captured separately and reported, because it carries the failure count and flows route on it. 99 — the gates could not run at all — is reported distinctly from a gate failure, so a configuration error can never read as a clean tree.

Verification — PHP 8.3, fresh from-lock install

Run in a php:8.3-cli container (+ git, python3, node, ext-xsl, ext-intl, ext-zip) with vendor/ removed and composer install run from the lock. No vendor/ was copied from any shared dev checkout.

Baseline — composer check:strict → exit 0

[hydra-gates] Base ref: origin/development (auto-detected) = 0eafa1165
[hydra-gates] Scope: diff vs origin/development — 2 changed file(s)
[hydra-gates] COVERAGE: 59 of 61 declared gates reported a result.
[hydra-gates] RESULT: ALL GATES PASSED — EXCEPT GATES 24 33, WHICH DID NOT RUN.
ALL CHECKS PASSED — read the gate COVERAGE block above before treating this as complete.

Positive control — injected lib/GateControl.php with error_log() (gate-2) and src/gateControl.js with window.confirm() (gate-34). Both are violations only the gates catch, so the signal cannot come from phpcs/psalm/phpstan. composer check:strict → exit 1:

[hydra-gates] Scope: diff vs origin/development — 4 changed file(s)
[gate-2] forbidden-patterns: FAIL — 1 forbidden calls
[gate-34] window-confirm: FAIL — 1 native dialog call(s) — use NcDialog / CnFormDialog
[hydra-gates] RESULT: 2 GATE(S) FAILED. Exit code is the failure COUNT, not a boolean.
SOME CHECKS FAILED (see above)
  gates: 2 gate(s) failed — run 'composer gates' directly; its exit code is the failure COUNT.

The injected files are confirmed present in the output (lint and phpcs both report on them), and per-section attribution confirms gates was the only failing check — phpcs emitted warnings only, which do not fail it.

Reverse direction — reverted the control commit, composer check:strict → exit 0, 59/61 coverage, no FAIL lines.

Unresolvable base ref — HYDRA_GATE_BASE_REF=origin/definitely-not-a-branch composer gates → exit 99, zero gate lines emitted, no green printed:

[hydra-gates] FATAL: $HYDRA_GATE_BASE_REF='origin/definitely-not-a-branch' does not resolve.
[hydra-gates] Refusing to scope to an empty set. NOTHING WAS CHECKED.

and through check:strict:

  gates: COULD NOT RUN (exit 99) — a configuration error, NOT a clean run. Nothing was gated.

Two things worth knowing

92 files here carry an @e2e exclude <reason> tag, plus 1 @spec exclude. The gate run now states this on every green, so a green earned by passing is distinguishable from one earned by waiving. Worth a look independently of this PR.

phpmd.baseline.xml — this repo uses the same baseline mechanism that was measured to suppress nothing in decidesk while reading as protection. Not touched here, but flagged.

Provisional

The dependency is declared against a path repository at ../hydra — the fleet's checkout layout. How hydra-gates should ultimately be distributed is an open decision (see hydra#504); a path repository is the shape that is verifiable today without one. If distribution lands differently, only the repositories block here changes.

Forge

Opened on GitHub, which is canonical for openbuild: ConductionNL/openbuild is public, its default branch is development, it is 232 commits ahead of the Codeberg copy (7 behind the other way — genuinely forked), it was pushed today, and it is the forge carrying the Actions workflows including pre-merge-check-strict. Codeberg's last activity was 2026-07-26.

@juanclaude-conduction
build(quality): run the Hydra gates as part of composer check:strict
2911279 
openbuild is the pilot for conduction/hydra-gates. The gates have until now only
run inside hydra's own containers, which means an agent finishing a task here
could report done without any of the 61 mechanical gates having looked at its
diff. Wiring them into check:strict makes "gates pass" part of the definition of
done in the same command everything else already runs.

The gates are diff-scoped per ADR-020, so this does NOT import openbuild's
inherited debt. A full-repo run of this tree fails 16 gates today (img-alt,
button-name, table-headers, spec-anchor-existence and friends); scoped to a PR's
own diff, only what that PR touched is enforced. `composer gates:full` is
available for the audit view, and is deliberately not what check:strict runs.

check:strict keeps its 0/1 contract, but the gate exit code is preserved
separately and reported, because it carries the failure COUNT and flows route on
it. 99 is distinguished from a gate failure in the summary: it means the gates
could not run at all, which is a configuration error and not a clean tree.

The dependency is declared against a `path` repository at ../hydra, which is the
fleet's checkout layout. That is the part of this change that is provisional —
see the PR description; how hydra-gates should be DISTRIBUTED (private Packagist,
a Satis mirror, or a split public repo) is an open decision, and a path
repository is the shape that is verifiable today without one.
@juanclaude-conduction
build(quality): lock conduction/hydra-gates at dev stability
37da205 
A path repository derives its version from the checked-out branch, so the
default minimum-stability of 'stable' rejects a '*' constraint outright:
'found conduction/hydra-gates[dev-main, ...] but it does not match your
minimum-stability'. '@dev' is the constraint that resolves, and it keeps
working once the package is eventually tagged.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ 24e6a4d

Check PHP Vue Security License Tests
lint ❌
phpcs ❌
phpmd ❌
psalm ❌
phpstan ❌
phpmetrics ❌
eslint ✅
stylelint ✅
composer ❌ ❌
npm ✅ ✅ 662/662
PHPUnit ⏭️
Newman ⏭️
Playwright ⏭️

Quality workflow — 2026-08-03 09:29 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde marked this pull request as draft August 3, 2026 09:31
@rubenvdlinde

rubenvdlinde commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

⚠️ Converted to draft — this is blocked on a decision, and CI just proved it

Do not merge. The gate wiring itself is verified working (see the PR body: both directions of the positive control, exit-code-is-count, loud unresolvable base, all on a fresh from-lock install in a PHP 8.3 container). What is not working is the delivery of the package, and CI caught it immediately.

The evidence

10 checks failed on this PR. All of them are the jobs that run composer install:

##[error]Source path "../hydra" is not found for package conduction/hydra-gates
  • quality / PHP Quality (lint, phpcs, phpmd, phpstan, psalm, phpmetrics) — fail
  • quality / License (composer), quality / Security (composer), quality / Quality Report, exporter-e2e — fail
  • Everything that does not touch composer (eslint, stylelint, Frontend Tests (unit), License (npm), Security (npm), lint-check, Features Check) — pass

This is a regression introduced by this PR, not pre-existing debt. The last three code-quality.yml runs on development all concluded success (9325bb17, 9f3f9297, 0eafa116).

Why

The dependency is declared against a path repository at ../hydra. That is true on a developer machine and in the dev containers, where the fleet checks every app out as a sibling of hydra. It is not true in a GitHub Actions runner, which checks out one repo.

What I am deliberately not doing

Three "fixes" are available and all of them are worse than stopping:

  1. Make the gates script exit 0 when the package is absent. This is precisely the || echo '…skipping' defect that this entire piece of work exists to remove. check:strict would go green having gated nothing.
  2. Clone hydra into ../hydra in CI. hydra's GitHub repo is private, so this needs a credential with read access to private ConductionNL source. openbuild's workflows currently reference exactly one secret, PROJECT_TOKEN, used twice, and there is no existing cross-repo checkout anywhere in .github/workflows/. Repurposing or minting a token for private source access is a new decision, not an implementation detail.
  3. Vendor the gate scripts into this repo. Creates a second source of truth for a 3,652-line runner. Drift in a gate runner is how a gate silently stops gating.

The decision needed

How conduction/hydra-gates gets distributed — tracked in ConductionNL/hydra#504. The realistic options:

  • a private Packagist / Satis mirror (needs COMPOSER_AUTH in every consuming repo's CI)
  • split the gates into their own public repo (a plain public vcs repository, no credentials anywhere — but a second repo to keep in sync with hydra's scripts/)
  • make hydra public (simplest consumer story; a separate call entirely)
  • keep it path-only and accept that the gates run locally and in the dev containers but not in hosted CI

I have not picked one. Once it is picked, the only thing that changes in this PR is the repositories block.

@rubenvdlinde rubenvdlinde mentioned this pull request Aug 3, 2026
Merged
rubenvdlinde added a commit to ConductionNL/.github that referenced this pull request Aug 3, 2026
@rubenvdlinde
feat(gates): host conduction/hydra-gates here, and wire it into the s…
fdad254 
…hared quality workflow (#131)

* feat(gates): host conduction/hydra-gates here, and wire it into the shared quality workflow

The 61 mechanical gates were merged into hydra as a composer package
(ConductionNL/hydra#504), but the pilot (ConductionNL/openbuild#104) could
not ship: the package was consumed through a `path` repository at `../hydra`,
which works on a developer machine and fails in CI with
`Source path "../hydra" is not found`. hydra is private, so a `vcs` repository
pointed at it would need credentials in every consuming repo and every runner.

This repository is public and already owns the shared workflows, so hosting
the package here solves distribution AND puts the gates in the default checks.

## What is here

`hydra-gates/` is now the SINGLE SOURCE OF TRUTH for the gate runner, its 24
Python/JS helpers, the vendored manifest + icon schemas, and the distributable
`bin/hydra-gates` entry point. The root `composer.json` names the package, so
a plain `vcs` repository resolves it — no subdirectory support required.

hydra delegates; it keeps no copy. Two copies that drift is the failure these
gates exist to catch.

Git history was NOT imported. A subtree split would have published the private
orchestrator's commit history into a public repository. Provenance is recorded
in hydra-gates/README.md instead.

## Wiring into quality.yml — blast radius

New `hydra-gates` job, gated on a new `enable-hydra-gates` input that
**defaults to false**. Nothing changes for any repo until that repo opts in.

This was deliberate. The gates are diff-scoped per ADR-020, but switching a
hard gate on fleet-wide turns every repo red whose in-flight branches happen to
touch files carrying inherited debt — openbuild alone fails 16 gates on a
full-repo run. Staged opt-in lets each repo enable it once its own diffs are
clean. Two further inputs, `hydra-gates-ref` (pin the gate version) and
`hydra-gates-require-full-coverage`, are also opt-in.

When enabled the job joins `report`'s `needs`, so a gate failure blocks the
Quality Report rather than being a green box nobody reads.

## The properties that had to survive, and how they are held

- **Exit code is the failure COUNT.** The run step captures it through an
  `&& / ||` list, because a `run:` block's default shell is `bash -e` and a
  bare call followed by `RC=$?` would abort before the assignment — collapsing
  the count to a boolean. That is one character away at all times.
- **Fail-closed base ref.** On a pull_request the job names the PR's real
  target branch. `@{upstream}` is not used anywhere (on a pushed feature
  branch it is the branch ITSELF, so the diff is empty exactly when CI runs —
  this produced "58 gates green over 0 changed files"). There is no `HEAD~1`
  fallback. An unresolvable base exits 99, prints no green, and the job
  reports 99 distinctly from "N gates failed".
- **`fetch-depth: 0`** on the app checkout is required, not an optimisation: a
  shallow clone has no base branch and the run would (correctly) refuse.
- **Coverage is asserted, not assumed.** The wrapper reads the gate inventory
  out of the runner rather than hardcoding 61, so gate 62 cannot go silently
  unmeasured.
- **No `|| echo '...skipping'` anywhere.** Missing prerequisites are named
  along with what they left uncovered.

## Verification

`.github/workflows/hydra-gates-package.yml` makes the proof permanent rather
than a one-off. It installs the package THROUGH COMPOSER from this repository's
public VCS URL into a scratch project in a clean `php:8.3-cli` container — no
local checkout of the package on the box, no credentials — and then runs the
positive control in both directions against a fixture repo:

- inject `error_log()` (gate-2) and `window.confirm()` (gate-34), violations no
  other fleet tool reports, and assert BOTH offending files are named, BOTH
  gates are named, and the exit code is exactly 2 — the count;
- remove them, and assert via `git ls-files --error-unmatch` that the revert
  actually applied before re-measuring, then assert exit 0 with coverage and
  waiver counts stated;
- unresolvable base ref -> exit 99, no green, no gate lines emitted.

node is deliberately absent from that container, so the run also has to name
the gates that left uncovered instead of counting them toward the green.

* fix(ci): correct the positive-control attribution, and clear ShellCheck

## The positive control asserted against the wrong stream

The install-from-published-location job asserted that the injected files were
named on stdout. They are not, by design: the runner prints a COUNT per gate
and writes the file list to /tmp/hydra-gate-<name>.log. The run itself was
correct — installed from the public VCS URL with no credentials, gate-2 and
gate-34 both fired, exit code 2 = the count — but the assertion was measuring
a stream that never carried the answer.

Now checked where the attribution actually lives, and strengthened: a gate that
reports FAIL while writing NO evidence log is also a failure, because "the gate
fired" and "the gate fired on what I injected" are different claims and only
the second one makes this a control.

## ShellCheck

This job was already failing for any PR touching a .sh file, on three
pre-existing scripts. Fixed rather than suppressed:

- docker/mail/seed-mail.sh — `email_data` (and the `cc_header` feeding it) were
  dead. Verified the python block below builds the message itself and reads
  $cc directly for both the header and the recipient list, so removing them
  changes nothing that is sent.
- docker/mail/seed-ox.sh — OX_ADMIN / OX_ADMIN_PASS were unread: every command
  authenticates as the CONTEXT admin. That made them a hardcoded master
  password sitting in a public repo for no reason. Removed.
  Also replaced `if [ $? -eq 0 ]` with a direct `if docker exec ...; then` —
  the two were separated by a blank line, so anything inserted between them
  would silently have become the command whose status was read.
- docker/test-database-compatibility.sh — quoted a comparison, and split two
  `local x=$(...)` assignments, which had been masking the pipeline's exit
  status behind `local`'s always-success.

The imported gate runner had never been ShellChecked at all — hydra has no
GitHub Actions — so moving it here surfaced 22 note/warning findings. They are
suppressed per-file with the reasoning recorded inline, NOT in a root
.shellcheckrc, which would have switched those checks off for every script in
this repository including ones written later. Rewriting 11 `sed` calls inside
gate logic that has no unit tests of its own is a change to what the gates
decide, and it belongs in its own PR with the gate output diffed either side.

One real fix in that set: tests/test-hydra-gates-bin.sh now does `cd || exit 1`.
Without it, a failed cd builds the fixture in the current directory and every
assertion measures the wrong repository while still looking like a normal run.
@rubenvdlinde
Merge remote-tracking branch 'origin/development' into feat/wire-hydr…
690c8a5 
…a-gates-check-strict
@rubenvdlinde
build(quality): resolve hydra-gates from the public package, not a pa…
ce22025 
…th repo

The `path` repository at `../hydra` is what blocked this PR. It works on a
developer machine and fails in CI with `Source path "../hydra" is not found`,
because the fleet's sibling-checkout layout does not exist in a CI job and
hydra is private, so no runner can fetch it.

conduction/hydra-gates now lives in ConductionNL/.github, which is PUBLIC and
already owns the shared workflows (ConductionNL/.github#131). Pointing at it
needs no credentials in this repo or on any runner.

- repositories: `path ../hydra` -> `vcs https://github.com/ConductionNL/.github.git`
  with `"no-api": true`, so composer clones over git instead of the GitHub API
  and a rate-limited unauthenticated runner cannot become a failed install.
- require-dev: `@dev` -> `^1.0`, resolving to the v1.0.0 tag. That also drops
  the dev-stability requirement the previous commit needed, because a path
  repository derives its version from the checked-out branch and a tag does not.
- composer.lock pins commit fdad2546f2ac68aa64be5fecef898784c1847538.

Nothing else changes. `composer gates`, `composer gates:full` and the gate
handling inside `check:strict` are untouched, including the part that captures
the gate exit code separately: it carries the FAILURE COUNT, and 99 ("could not
run at all") is still reported distinctly from a gate failure so a
configuration error can never read as a clean tree.

Also merged origin/development in. The branch predated the fleet-wide
timeout-minutes work, so it was carrying a silent revert of the bounds on
exporter-e2e.yml and pull-request-lint-check.yaml.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ 8cca9ef

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
composer ✅ ✅ 101/101
npm ✅ ✅ 662/662
PHPUnit ✅
Newman ⏭️
Playwright ⏭️

Coverage: 89.4% (17/19 statements)

Quality workflow — 2026-08-03 18:45 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
ci(quality): actually run the gates in CI, not just in a local compos…
ad63f34 
…er script

Wiring the gates into `composer check:strict` was a local-only change. Nothing
in this repository's CI invokes check:strict — no workflow does, verified by
grepping .github/workflows — so this PR would have merged green with the 61
gates never having executed on any diff. A gate that runs only on a developer's
machine is not a gate, and a green that never ran it is the failure mode the
gates exist to catch.

The shared quality workflow now ships a `hydra-gates` job (ConductionNL/.github#131),
opt-in per repo and defaulting to off. openbuild is the pilot, so it opts in.

`hydra-gates-ref` is pinned to the v1.0.0 tag rather than tracking `main`, so a
change to the gates cannot move this repo's verdict without a commit here to
move the pin.

The job resolves the diff base from the PR's real target branch, checks out the
app with fetch-depth: 0 (a shallow clone has no base to diff against), and
reports exit 99 — "could not run at all" — distinctly from "N gates failed".
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ e3f8ad5

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
composer ✅ ✅ 101/101
npm ✅ ✅ 662/662
PHPUnit ✅
Newman ⏭️
Playwright ⏭️

Coverage: 89.4% (17/19 statements)

Quality workflow — 2026-08-03 18:49 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde rubenvdlinde mentioned this pull request Aug 3, 2026
Merged
@rubenvdlinde
build(quality): bump hydra-gates to v1.0.1 for the gate-4 audit fix
50d3f72 
The first real CI run of the gates on this PR failed on gate-4:
`composer-audit: FAIL — CVEs or advisories`. There are no CVEs — auditing this
repo's lock properly reports "No security vulnerability advisories found."

`composer audit` with no vendor/ present does not audit the lock. On the
runner's composer it errors and exits non-zero, which gate-4 reported as a
security finding. On composer 2.7.x the same situation prints "No packages -
skipping audit" and exits 0 — a silent fail-open that passes having audited
nothing.

Fixed in the package (ConductionNL/.github#135, released as v1.0.1): gate-4
audits `--locked`, treats an exit 0 that audited no packages as a failure, and
reports "could not run" in those words rather than as a CVE finding.

Both the workflow pin and composer.lock move to v1.0.1 so the two paths into
the gates cannot disagree about which version this repo is on.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ 14d8c03

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
composer ✅ ✅ 101/101
npm ✅ ✅ 654/654
PHPUnit ✅
Newman ⏭️
Playwright ⏭️

Quality workflow — 2026-08-03 20:51 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde marked this pull request as ready for review August 3, 2026 20:53
@rubenvdlinde
rubenvdlinde merged commit c7598bf into development Aug 3, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@rubenvdlinde @juanclaude-conduction