Skip to content

fix(ci): make composer check:strict able to fail on tests - #196

Merged
rubenvdlinde merged 1 commit into
developmentfrom
fix/check-strict-can-fail
Aug 2, 2026
Merged

fix(ci): make composer check:strict able to fail on tests#196
rubenvdlinde merged 1 commit into
developmentfrom
fix/check-strict-can-fail

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Problem

test:unit and test:all ended in:

./vendor/bin/phpunit --colors=always || echo 'Tests require Nextcloud environment, skipping...'

phpunit's exit status was discarded unconditionally, so composer check:strict could never fail on a test failure — and this repo's check:strict passes everything else, so the whole command was green with the test gate switched off.

Here the excuse in the message turns out to be true: with a freshly installed vendor/ the suite still dies with Class "OC\Mail\EMailTemplate" not found (an internal OC\ class that only exists inside a running Nextcloud). But the mask was never conditional on that, so a genuine failure inside CI — where the tree is present — looked identical to "no Nextcloud here".

Change

composer.json only. Replace the blanket mask with the actual precondition, plus --no-coverage:

if [ ! -f vendor/bin/phpunit ]; then echo 'SKIPPED: phpunit not installed - run composer install';
elif [ ! -f ../../lib/base.php ]; then echo 'SKIPPED: the unit suite needs a Nextcloud server tree (../../lib/base.php not found) - run from inside a Nextcloud checkout or in CI';
else ./vendor/bin/phpunit --colors=always --no-coverage; fi

The plain if [ -f vendor/bin/phpunit ] guard used elsewhere in the fleet is deliberately not used here: it would make check:strict permanently, falsely red on every machine outside a Nextcloud tree, which is the fastest way to get a gate ignored again.

Positive control — measured on this tree

PHP 8.3.32 container, vendor/ freshly installed, app mounted at /nc/apps-extra/nldesign so ../../lib/base.php is under my control:

state composer check:strict
no server tree exit 0 — every tool runs and passes; test:all prints SKIPPED: the unit suite needs a Nextcloud server tree
server tree present exit 1 — phpunit runs and its non-zero status propagates, check:strict naming test:all (error code 255)
old composer.json, same phpunit failure test:all never named — swallowed by || echo

Honest caveat: in the middle row the "server tree" is a minimal stand-in defining OC_App/OC_Hook, which is enough for phpunit to start but not enough for this suite to pass — so what that row proves is that phpunit's exit status now propagates, not that a specific injected assertion failed. A real Nextcloud tree (which CI provides) is needed for the suite itself.

Policy / state after this PR

Tooling only — no findings are fixed here. check:strict exits 0 outside a Nextcloud tree (all of lint, phpcs, phpmd, psalm, phpstan pass, test:all skips loudly). The real follow-up this exposes is that the unit suite cannot run standalonetests/Unit/Controller/SettingsControllerAuditTest.php and others reach for internal OC\ classes. That was completely invisible while the mask was in place.

test:unit and test:all ended in '|| echo Tests require Nextcloud
environment, skipping...', discarding phpunit's exit status
unconditionally. Every other tool in check:strict passes here, so the
whole command was green with the test gate switched off.

The excuse in that message is actually TRUE for this repo - with a
freshly installed vendor/ the suite still dies on 'Class OC\Mail\EMailTemplate
not found' - but the mask was not conditional on it, so a genuine failure
inside CI looked identical to 'no Nextcloud here'.

Guarded on the real precondition instead, plus --no-coverage.

Positive control (PHP 8.3.32 container, app mounted at /nc/apps-extra/nldesign):
  no server tree      -> exit 0, loud SKIPPED, test:all not named
  server tree present -> exit 1, check:strict NAMES test:all (code 255)
  old composer.json   -> same phpunit failure, test:all never named

Tooling only.
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/nldesign @ 31b0290

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
composer ✅ 100/100
npm
PHPUnit ⏭️
Newman ⏭️
Playwright ⏭️

Quality workflow — 2026-08-02 22:18 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde merged commit 922ebae into development Aug 2, 2026
20 of 27 checks passed
@rubenvdlinde
rubenvdlinde deleted the fix/check-strict-can-fail branch August 2, 2026 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant