Skip to content

chore(quality): make composer check:strict an honest gate - #14

Merged
rubenvdlinde merged 1 commit into
developmentfrom
chore/measure-strict-gate-2026-08-02
Aug 2, 2026
Merged

chore(quality): make composer check:strict an honest gate#14
rubenvdlinde merged 1 commit into
developmentfrom
chore/measure-strict-gate-2026-08-02

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Make composer check:strict an honest gate

This repo's check:strict was not a working gate. This PR repairs it and reports exactly what the repaired gate surfaces.

Analysis-path verification (before believing any number)

All analyzers were run inside nextcloud:32-apache, which matches the CI php-version: 8.3 pin. Host PHP 8.2 is too old; PHP 8.5 (nextcloud:34) crashes Psalm 5.26.1 in its own MethodCallAnalyzer.

  • php -vPHP 8.3.32
  • php -mimagick present (a missing imagick alone has shifted psalm counts elsewhere in the fleet)
  • vendor/ was built by a clean composer install from composer.lock (49 packages), not copied from a shared checkout.
  • OCP stubs: vendor/nextcloud/ocp/OCP is a real directory, 132 entries / 960 PHP files, not a symlink and with no OCP.bak beside it. Verified require 'vendor/nextcloud/ocp/OCP/IRequest.php'; interface_exists('OCP\IRequest')true.
    • Note: interface_exists('OCP\IRequest') via the Composer autoloader is false, and that is correct — nextcloud/ocp declares no autoload section; it is a stubs-only package consumed through phpstan scanDirectories / psalm extraFiles. This is not the "empty stub dir" failure mode.
  • ./vendor/bin/phpmd exists. The shipped script invoked bare phpmd, which does not resolve (127 command not found).

Measured BEFORE any change

script command as shipped raw exit swallowed by || echo? effect on check:strict
lint find … -print0 | xargs -n1 php -l 0 n/a passes (1 file)
phpcs ./vendor/bin/phpcs --standard=phpcs.xml 3 no fails the gate
phpmd phpmd lib text phpmd.xml 127 (phpmd: not found) yes silently passes
psalm ./vendor/bin/psalm --threads=1 --no-cache 1 yes silently passes
phpstan ./vendor/bin/phpstan analyse --memory-limit=1G 1 yes silently passes
test:all ./vendor/bin/phpunit --colors=always 1 yes silently passes

Correction to the premise this work started from. The gate was not "green but dead" — it was red. phpcs carries no || echo, so composer check:strict has been exiting 1. Nobody noticed because no CI workflow in this repo invokes check:strict, phpcs, psalm, phpstan or phpmd at all; the only composer call anywhere in .github/workflows/ is composer install --no-dev in the three release workflows.

Not one of those failures was a code finding. Every one was a configuration error with the same root cause.

Root cause: the tools were aimed at a directory that does not exist

This is a Python ExApp sidecar wrapper. ex_app/lib/main.py + requirements.txt + Dockerfile are the app. There is no lib/ directory, yet phpcs.xml (<file>lib</file>), psalm.xml (<directory name="lib"/>) and the phpmd script (phpmd lib …) all pointed at it, and phpstan had no config at all.

The entire PHP toolchain was copied from the PHP-app template. The only PHP this repo authors is one file: phpcs-custom-sniffs/CustomSniffs/Sniffs/Functions/NamedParametersSniff.php (408 lines) — the sniff enforcing our named-parameter rule. Because phpcs was aimed at lib, that sniff has never once been linted, and the sniff itself has never run on anything in this repo.

PHPStan decision: APPLICABLE — config added

lib/ is absent, but PHP is present (the 408-line sniff), and it is real logic worth analysing. Adding phpstan.neon costs nothing at runtime and gives the sniff genuine level-5 coverage, so phpstan is not marked N/A here. The config is deliberately minimal: level: 5, paths: [phpcs-custom-sniffs], scanDirectories: vendor/nextcloud/ocp. None of openregister's app-specific ignoreErrors were copied — none of them apply, and importing them would pre-suppress findings this repo has not earned.

Both configs carry a comment: add lib back the moment PHP app code lands here.

Counts the repaired gate surfaces, and how each was resolved

Symbol-source cascades were eliminated before counting — the raw numbers were badly inflated:

  • phpstan: 52190. 52 was mostly "unknown class PHP_CodeSniffer\…"; 19 was PHPCS's T_* token constants, which src/Util/Tokens.php creates at runtime via define(). squizlabs/php_codesniffer also declares no Composer autoload section, so neither symbol set is visible without help.
  • psalm: 1 (UndefinedClass … Sniff) → 19 (UndefinedConstant T_*, exposed once the class resolved) → 0.

analysis-bootstrap.php (new) supplies those symbols for both tools — PHPStan bootstrapFiles, Psalm autoloader. This adds the missing symbol source; it does not suppress the diagnostics. Suppressing them by rule name would have hidden real findings behind the same names.

tool findings on the real target outcome
phpcs 3, then 2 more once the docblock parsed all 5 fixed, 0 baselined
phpmd 4 2 fixed, 2 baselined (with reasons below)
psalm 0 after symbol sources supplied clean
phpstan 0 at level 5 clean

phpcs — all fixed, nothing suppressed:

  1. CRLF line endings on the whole file (auto-fixed via the repo's own phpcbf).
  2. @package must precede @author.
  3. @author must be Display Name <email>.
  4. Missing @license — surfaced only after (2) and (3) were fixed and PEAR could parse the block.
  5. @license must carry a URL and a name.

phpmd — 2 fixed: ShortVariable ×2 ($j$ptr, two local token cursors). Plain renames, no behaviour change.

Baselines and their reasons

phpmd.baseline.xml contains exactly two entries, both complexity/style, neither a defect:

  1. WeightedMethodCountNamedParametersSniff class complexity 65 vs threshold 50.
  2. CyclomaticComplexityhasUnnamedArguments() complexity 20 vs threshold 15.

Reason: both are inherent to hand-written PHP token-stream parsing, which is unavoidably branchy. Reducing them means restructuring the control flow of the sniff that enforces a fleet-wide lint rule — a behavioural risk that does not belong in a gate-repair PR whose whole point is that this file has never been linted. Tracked in the issue below rather than silently suppressed. Both entries are pinned to a specific rule + file (+ method), so any new complexity violation elsewhere still fails the gate.

No psalm baseline and no phpstan baseline were created — neither tool needed one.

Issue filed: #13 — tracks removing both baseline entries (and deleting phpmd.baseline.xml once empty).

test:all deliberately left dead — and its message corrected

test:all keeps its || echo fallback. But the cost was measured, and the shipped message was false: phpunit exits 1 here because there is no phpunit.xml and no tests/ directory — not because "tests require a Nextcloud environment". The message now says so. Behaviour is unchanged; turning this into a real gate needs a test suite to exist first.

composer.json shape chosen: plain ./vendor/bin/X (shape b)

Matching hermiq / hrmq / portaliq / petstore / nldesign / nc-app-template rather than the if [ -f … ] guard used by shillinq and openregister.

Why: all four binaries are hard require-dev dependencies and are present after composer install. A tolerant if [ -f … ] guard would reintroduce exactly the failure this PR fixes — an absent tool passing silently — and the original defect here was a bare phpmd that was never on PATH, which a file-existence guard would have kept invisible. If the tool is missing, the correct outcome is a loud failure.

phpmd's bare binary is now ./vendor/bin/phpmd regardless of shape. The same bare-binary / missing-lib/ bug was also fixed in the non-gate helpers quality:phpcs-score, quality:phpmd-score, phpmetrics, phpmetrics:violations.

Result

composer check:strictALL CHECKS PASSED, exit 0, with lint, phpcs, phpmd, psalm and phpstan all genuinely executing against real code and passing on their merits.

Known gaps, stated plainly

  • The gate still runs nowhere in CI. This PR makes check:strict correct; it does not wire it into a workflow. Until that happens the repair is only enforced locally. This is the single highest-value follow-up.
  • NamedParametersSniff.php is byte-identical across openklant, opentalk, openzaak and valtimo — so is the debt in it. It should live in one shared package instead of four copies.
  • The autoload.psr-4 map still points OCA\Valtimo\ at the non-existent lib/. Left as the forward-looking declaration for when PHP app code lands.
  • phpqa / phpqa:full / phpqa:ci scripts reference a phpqa binary that is not in require-dev and cannot run. Not touched — out of scope for this gate.

The strict gate did not work. Measured on nextcloud:32-apache (PHP 8.3.32,
imagick), matching the CI php-version 8.3 pin, against a clean composer
install from composer.lock:

  lint      exit 0
  phpcs     exit 3   NOT swallowed - the gate has been RED, not green
  phpmd     exit 127 'phpmd: not found' (bare binary), swallowed by || echo
  psalm     exit 1   swallowed by || echo
  phpstan   exit 1   swallowed by || echo (no config at all)
  test:all  exit 1   swallowed by || echo

Every failure was a configuration error, not a code finding: this is a
Python ExApp sidecar wrapper and lib/ does not exist, yet phpcs.xml,
psalm.xml and the phpmd script all pointed at it. No CI workflow invokes
check:strict, so nobody saw the red.

The only PHP this repo authors is the 408-line NamedParametersSniff, which
had therefore never been linted. Point the tools at it:

- phpcs.xml / psalm.xml / phpmd + phpstan paths -> phpcs-custom-sniffs
- add phpstan.neon (level 5) - phpstan is applicable, not N/A: there is
  real PHP here and the config costs nothing
- add analysis-bootstrap.php supplying OCP/NCU and PHP_CodeSniffer symbols
  (neither package declares a composer autoload section). This removed a
  52 -> 19 -> 0 phpstan cascade and a 1 -> 19 -> 0 psalm cascade by
  supplying the missing symbol source, not by suppressing diagnostics.
- drop the || echo fallbacks from phpmd/psalm/phpstan; use ./vendor/bin/X

Findings on the real target, and what was done with each:

  phpcs   5 (CRLF line endings, @package/@author order, @author format,
            missing @license, @license needs a URL) - ALL FIXED
  phpmd   4 - 2 fixed ($j -> $ptr), 2 baselined (class complexity 65>50,
            hasUnnamedArguments 20>15); both are complexity/style inherent
            to token-stream parsing, tracked in an issue, not defects
  psalm   0
  phpstan 0

Positive control: a probe method with var_dump, a short variable and a call
to an undefined method makes all four analyzers fail, and the phpmd baseline
does not mask it - so the green result is not vacuous.

test:all is left dead on purpose (no phpunit.xml, no tests/), but its skip
message no longer claims a Nextcloud environment is the reason.
@rubenvdlinde
rubenvdlinde merged commit dc76797 into development Aug 2, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant