Skip to content

fix(ci): make composer check:strict able to fail on tests - #15

Merged
rubenvdlinde merged 1 commit into
developmentfrom
fix/check-strict-can-fail
Aug 2, 2026
Merged

fix(ci): make composer check:strict able to fail on tests#15
rubenvdlinde merged 1 commit into
developmentfrom
fix/check-strict-can-fail

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Problem

test:unit and test:all ended in:

./vendor/bin/phpunit --colors=always || echo 'No PHP test suite in this repo (no phpunit.xml, no tests/) - skipping...'

phpunit's exit status was discarded unconditionally. The message happens to be true today — this repo is a Python ExApp wrapper with no PHP test suite — but the mechanism never checked that. It swallowed whatever phpunit did, so a real failure (now, or the day a test suite is added here) would look exactly like "no tests to run", and composer check:strict could not fail on tests.

That matters because check:strict is about to be the thing an agent checks before deciding its job is done. A step that cannot fail satisfies that instruction with a no-op.

Change

composer.json only. Guard on the actual precondition instead of masking the exit code:

if [ -f phpunit.xml ] || [ -f phpunit.xml.dist ]; then ./vendor/bin/phpunit --colors=always; else echo 'SKIPPED: no phpunit.xml in this repo - there is no PHP test suite to run'; fi

A genuinely absent suite is now a stated fact in the output, and a present-but-failing suite fails the gate.

Positive control — measured on this tree

PHP 8.3.32 container, vendor/ freshly installed from composer.lock, injecting a temporary phpunit.xml plus one deliberately failing test:

state composer check:strict
old composer.json + failing test exit 0ALL CHECKS PASSED; the failure printed and was swallowed by || echo
new composer.json + failing test exit 1SOME CHECKS FAILED, naming test:all (Tests: 1, Assertions: 1, Failures: 1)
new composer.json, test removed exit 0 — clean, with SKIPPED: no phpunit.xml in this repo

The temporary phpunit.xml and control test are not part of this PR — the diff is composer.json only.

State after this PR

composer check:strict exits 0 on this branch: lint, phpcs, phpmd, psalm and phpstan all pass, and test:all skips loudly. No findings needed fixing here.

Note on measurement

The first measurement round in this repo showed psalm failing with Cannot resolve stubfile path .../OCP/Capabilities/ICapability.php. That turned out to be an artefact of the shared local dev checkout (its vendor/nextcloud/ocp/OCP had been renamed to OCP.bak), not a repo defect. All numbers above are from a clean composer install.

test:unit and test:all ended in
  ./vendor/bin/phpunit --colors=always || echo 'No PHP test suite in this repo ... skipping...'
so phpunit's exit status was discarded unconditionally. The message was
true today (this repo is a Python ExApp wrapper with no PHP test suite)
but the mechanism was not conditional on it: any real test failure, now
or after a test suite is added, would have been swallowed the same way.

Replaced with a guard on the actual precondition, so an absent suite is a
visible stated fact and a present-but-failing suite fails the gate:
  if [ -f phpunit.xml ] || [ -f phpunit.xml.dist ]; then ./vendor/bin/phpunit --colors=always; else echo 'SKIPPED: no phpunit.xml ...'; fi

Positive control, measured on this tree in a PHP 8.3.32 container with a
freshly installed vendor/, injecting a phpunit.xml plus one deliberately
failing test:
  old composer.json + failing test -> check:strict exit 0  (swallowed)
  new composer.json + failing test -> check:strict exit 1, naming test:all
  new composer.json, test removed  -> check:strict exit 0, loud SKIPPED

Tooling only.
@rubenvdlinde
rubenvdlinde merged commit dc7b2a9 into development Aug 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant