fix(quality): stop a phpmd finding from silently deleting the PHPUnit signal - #140
Conversation
… signal `phpunit` was gated on `needs.php-quality.result != 'failure'`. php-quality is a matrix job, so its result is 'failure' if ANY leg fails — including phpmd (complexity/mess detection) and phpcs (formatting). Neither tells you anything about whether the tests pass, but a red leg deleted the entire PHPUnit matrix. The loss is invisible by construction: a skipped matrix renders as one greyed "skipping" placeholder in the checks list, which reads like a non-event rather than a gate that stopped running. Measured 2026-08-04: ConductionNL/procest phpmd RED (25 findings) → 26 jobs, ZERO PHPUnit legs ConductionNL/decidesk phpmd green → 29 jobs, 4 green PHPUnit legs Same workflow inputs; the only difference was phpmd's colour. procest's 1684 tests had had no CI signal at all for as long as phpmd had been red. And decidesk's phpmd was green only because a phpmd.baseline.xml was suppressing 73 real findings — so whether its suite ran at all was contingent on a suppression file. The `security` gate is kept: a repo with a known-vulnerable dependency should not spin up servers. This aligns phpunit with the playwright job, which already deliberately declines to gate on php-quality. `needs:` is retained for ordering. Newman keeps its php-quality gate for now — it stands up a live stack and is a separate risk/cost decision; noted inline.
Live positive control — the fix works, and it immediately found something
Run 30866791400:
phpmd red, PHPUnit ran. That is exactly the behaviour the old What it surfaced — the gate was hiding a second, worse defectAll four PHPUnit legs failed: Zero failures, zero errors — 223 risky tests. procest's This is invisible locally: So procest's PHPUnit job was not merely dark — it was red on arrival. Two dead-gate causes stacked: the job was skipped, and had it not been skipped it could never have passed. The skip is what kept the second one invisible. For contrast, decidesk carries identical Consequence for merging this PRMerging turns procest's PHPUnit red where it is currently silent. That is the correct outcome and the entire point — but it should land together with, or just ahead of, the annotation fix, so procest is not left red on a newly-visible axis. Tracking that separately; this PR is what makes the defect visible at all. The probe branch |
…risky → 0 under coverage procest's PHPUnit legs had never run in CI. They were gated on `needs.php-quality.result != 'failure'`, and procest's phpmd leg has been red on 25 complexity findings, so `development` runs produced 26 jobs with a single greyed-out PHPUnit placeholder. ConductionNL/.github#140 removed that gate; the matrix expanded to 29 jobs and the suite failed on its first real run. The failure was invisible locally. `composer test:all` runs PHPUnit WITHOUT coverage, and all 1684 tests pass that way. CI runs `--coverage-clover`, and phpunit.xml sets `beStrictAboutCoverageMetadata="true"` + `failOnRisky="true"`. That combination only has teeth when a coverage driver is present: with covers metadata declared, PHPUnit marks a test risky if it executes production code that is listed neither as covered nor as used. 220 tests did. Every one of the 220 reported the same reason — "This test executed code that is not listed as code to be covered or used". None of them were missing `@covers`; they were exercising a collaborator the class docblock never declared: result value objects (ActionResult, GuardResult, BagLookupResult), domain exceptions (DecisionEvaluationException, HermiqAssistantException), the IntegrationMode enum, the SearchesObjects trait, and a handful of base classes and helper services. `@uses` is the correct declaration for all of them: it permits the collateral execution without attributing the code as covered, so reported coverage is unchanged (30.87% line coverage; the guard's baseline is 0.00 and passes). Nothing here weakens a check — `failOnRisky`, `beStrictAboutCoverageMetadata` and `--coverage-clover` are all untouched, and no test assertion changed. The risky set is execution-order-dependent: for a class whose only executed lines are load-time (enum cases, class constants), the attribution lands on whichever test loads it first, so consecutive local runs reported 227 and then 220. The declarations here were derived from the union of a full-suite run and a 245-file isolated sweep, then verified both ways: full suite Risky: 0 on two consecutive fresh-cache runs, and 0 risky pairs across all 245 files run in isolation. Tests: 1684, Assertions: 5614, Skipped: 5, Risky: 220 → 0. Exit 1 → 0.
… 0 (#716) procest's PHPUnit legs had never run in CI: they were gated on `needs.php-quality.result != 'failure'` and phpmd has been red, so development produced 26 jobs with one greyed-out placeholder. ConductionNL/.github#140 removed that gate and the suite failed on its first real execution with 223 risky tests. All 223 reported the same reason — 'executed code that is not listed as code to be covered or used'. None were missing @Covers; each was executing a collaborator its docblock never declared (result value objects, domain exceptions, the IntegrationMode enum, the SearchesObjects trait, base classes). Declared with @uses, which permits the collateral execution without attributing it as covered. failOnRisky, beStrictAboutCoverageMetadata and --coverage-clover are untouched; no assertion, fixture or lib/ file changed. 87 files, +249 lines, all docblock annotation under tests/Unit/. Verified: Tests: 1686, Assertions: 5632, Skipped: 5, Risky: 0 on all four legs (PHP 8.3/8.4 x NC stable31/stable32). Positive control pushed and reverted — a bogus assertion plus one removed @uses turned all four legs red naming both, and the revert was confirmed byte-identical to the pre-control tree.
The defect
phpunitwas gated onneeds.php-quality.result != 'failure'.php-qualityis a matrix job, so its result isfailureif any leg fails —lint,phpcs,phpmd,psalm,phpstan,phpmetrics. A complexity or formatting finding therefore deleted the entire PHPUnit matrix.This is a dead gate in the strict sense: its absence looks exactly like its success. A skipped matrix renders in the checks list as a single greyed-out
skippingplaceholder — not red, not missing, just quietly not there. Nothing in the PR view says "your tests did not run".The evidence — a natural A/B, measured 2026-08-04
ConductionNL/procest(run 30863839239)ConductionNL/decidesk(run 30862373575)Identical workflow inputs. The only difference is phpmd's colour.
Two things follow:
phpmd.baseline.xmlwas suppressing 73 real findings. So whether decidesk's suite ran at all was contingent on a suppression file. Delete the baseline honestly — which is the direction that repo is going — and its test signal would have gone dark too.That second point is the one that makes this structural rather than cosmetic: the fix for one problem (remove the suppression) silently caused another (lose the tests).
The change
securitygate kept. A repo with a known-vulnerable dependency should not spin up servers.needs:retained for ordering; with!cancelled()and no result condition, PHPUnit runs regardless of static-analysis colour.playwright, which in this same file already deliberately declines to gate on php-quality (itsif:checks onlyneeds.security.result). That inconsistency is itself evidence the php-quality gate on phpunit was an oversight rather than a decision.newmankeeps its php-quality gate. It stands up a live stack and is a separate cost/risk decision; left unchanged and noted inline.Verification
jobscount unchanged at 18;phpunit.ifandplaywright.ifare now equivalent in their php-quality treatment.ConductionNL/procest@ci/prove-phpunit-gatepointscode-quality.ymlat this branch while procest's phpmd is still red. If the fix works, that run shows phpmd failing and the PHPUnit legs executing. Result posted below as a comment.Blast radius
Fleet-wide (every Conduction app consumes
quality.yml@main). The effect is strictly more signal: repos whose php-quality is green see no change; repos whose php-quality is red gain back their test results. CI cost rises only for repos that are currently red.