fix(ci): make composer check:strict able to fail on tests - #101
Merged
Conversation
test:all (and test:unit where present) ended in '|| echo Tests require Nextcloud environment, skipping...', so phpunit's exit status was discarded unconditionally and check:strict could never fail on a test failure. This repo's suite runs standalone, so the message was untrue here. Positive control, PHP 8.3.32 container, vendor/ freshly installed, adding one deliberately failing test under tests/: old composer.json + failing test -> test:all never named, swallowed new composer.json + failing test -> check:strict fails NAMING test:all new composer.json, test removed -> test:all passes Also passes --no-coverage: this repo's phpunit.xml requests coverage, and without a driver PHPUnit warns and exits non-zero, which would have made the newly-live gate red for an environment reason rather than a code one. shillinq and decidesk already do this. Tooling only - no findings fixed here.
Contributor
Quality Report — ConductionNL/openbuild @
|
| Check | PHP | Vue | Security | License | Tests |
|---|---|---|---|---|---|
| lint | ✅ | ||||
| phpcs | ✅ | ||||
| phpmd | ✅ | ||||
| psalm | ✅ | ||||
| phpstan | ✅ | ||||
| phpmetrics | ✅ | ||||
| eslint | ✅ | ||||
| stylelint | ✅ | ||||
| composer | ✅ | ✅ 100/100 | |||
| npm | ✅ | ✅ 662/662 | |||
| PHPUnit | ✅ | ||||
| Newman | ⏭️ | ||||
| Playwright | ⏭️ |
Coverage: 89.4% (17/19 statements)
Quality workflow — 2026-08-02 22:14 UTC
Download the full PDF report from the workflow artifacts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
test:allandtest:unitended in:phpunit's exit status was discarded unconditionally, so
composer check:strictcould never fail on a test failure. The excuse in the message is not true for this repo — the suite runs standalone.This matters because
check:strictis about to be the thing an agent checks before deciding its job is done. A step that cannot fail satisfies that instruction with a no-op.Positive control — measured on this tree
PHP 8.3.32 container,
vendor/freshly installed fromcomposer.lock. One deliberately failing test added totests/:composer check:strictcomposer.json+ failing testALL CHECKS PASSED, with the failure printed right above it —test:allnever named,Tests require Nextcloud environment, skipping...printed insteadcomposer.json+ failing testtest:allcomposer.json, test removedThe control test is not in this PR — the diff is
composer.jsononly.Change
composer.jsononly, using the guard already shipped inshillinqandopenregister, plus--no-coverage:--no-coverageis deliberate: this repo'sphpunit.xmlrequests coverage, and without a coverage driver PHPUnit emits a runner warning and exits non-zero. Without the flag the newly-live gate would go red on any machine lacking pcov/xdebug — for an environment reason, not a code reason, which is the fastest way to get a gate ignored again.shillinqanddecideskalready pass--no-coveragehere.Policy / state after this PR
composer check:strictnow exits 0 on this branch and every tool really runs:lint,phpcs,phpmd,psalm,phpstanall pass andtest:allruns 754 tests / 2277 assertions green. Nothing needed fixing — but until this PR none of that was being asserted for tests.This gives the sharpest form of the control: exit 0 → exit 1 (injected failure) → exit 0 (reverted), on one tree.