Skip to content

fix(quality): remove PHP test scripts that manufacture a green in a repo with no PHP tests - #19

Merged
rubenvdlinde merged 1 commit into
developmentfrom
fix/remove-phantom-php-test-scripts
Aug 4, 2026
Merged

fix(quality): remove PHP test scripts that manufacture a green in a repo with no PHP tests#19
rubenvdlinde merged 1 commit into
developmentfrom
fix/remove-phantom-php-test-scripts

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Problem

This repo is a Python ExApp. The application is ex_app/lib/main.py, shipped via Dockerfile + entrypoint.sh + requirements.txt. The only PHP in the repo is phpcs-custom-sniffs/ (a single sniff) plus analysis-bootstrap.php.

Verified in this repo — none of these exist:

  • phpunit.xml, phpunit.xml.dist
  • tests/
  • pytest.ini, pyproject.toml, conftest.py
  • any test_*.py or *_test.py

composer.json nevertheless declared:

"test:unit": "if [ -f phpunit.xml ] || [ -f phpunit.xml.dist ]; then ./vendor/bin/phpunit --colors=always; else echo 'SKIPPED: no phpunit.xml in this repo - there is no PHP test suite to run'; fi",
"test:all":  "if [ -f phpunit.xml ] || [ -f phpunit.xml.dist ]; then ./vendor/bin/phpunit --colors=always; else echo 'SKIPPED: no phpunit.xml in this repo - there is no PHP test suite to run'; fi",

The condition is false by construction — there is no code path in this repo that could ever make it true. Both scripts were a guaranteed exit 0 that ran nothing, and check, check:full and check:strict all chained them, so every aggregate carried a test leg that could never fail. test:coverage and coverage:check (a 75% clover threshold) were also declared for a suite that does not exist.

A script that pretends to run a PHP test suite in a repo with no PHP application code is worse than no script: it manufactures a green. Removing it makes the absence of tests visible rather than papered over.

Change

Removed: test:unit, test:all, test:coverage, coverage:check.

Rewritten: check, check:full, check:strict — they no longer reference the removed scripts, and on success they now say what they did and did not cover:

ALL CHECKS PASSED - STATIC ANALYSIS ONLY.
This is a Python ExApp: the application lives in ex_app/lib/, and the only
PHP in the repo is phpcs-custom-sniffs/. There is NO automated test suite of
any kind - no phpunit.xml, no tests/, no pytest config, no test_*.py - and
the Makefile 'test' target is an interactive 'docker run -it' that asserts
nothing and cannot run in CI. This green therefore says NOTHING about
behaviour. Do not re-add a test:all script until a real suite exists.

check:strict now runs lint phpcs phpmd psalm phpstan only.

Verified nothing else calls the removed scripts

  • No reference to test:all, test:unit, test:coverage or coverage:check anywhere in .github/workflows/.
  • The shared ConductionNL/.github quality.yml never invokes composer test:* — its frontend-tests job reads package.json, not composer.json.
  • enable-phpunit defaults to false in that shared workflow and is not set by this repo, so no PHPUnit job existed in CI either. Nothing regresses.

Measured

After the change (openklant, composer 2 container — the four ExApp repos are byte-identical in this region and got the identical edit):

composer test:all      -> exit 1   ("test:all" is not defined)
composer check:strict  -> runs lint phpcs phpmd psalm phpstan only

composer validate passes on all four composer.json files.

Note: composer psalm currently fails in these repos with Cannot resolve stubfile path vendor/nextcloud/ocp/OCP/Capabilities/ICapability.php. That is pre-existing and environmental — I reproduced it on the untouched origin/development checkout with the original composer.json, so it is not caused by this PR. CI installs nextcloud/ocp and does not hit it.

Measured test count

Zero. That is the point of this PR. There is no test count to surface because there is no suite.

What this PR does NOT do

This repo still has no automated tests of any kind. The Makefile test target is:

test:
	docker run --rm -it \
	...

— an interactive container launch that asserts nothing and cannot run in CI.

I did not invent a Python test suite. Wiring up pytest (a pyproject.toml/pytest.ini, a tests/ package, fixtures for the ExApp's FastAPI surface, and a CI job to run them) is separate, larger work and is deliberately out of scope here. This PR only stops the repo from reporting a test result it never had.

…epo with no PHP tests

This repo is a Python ExApp. The application is ex_app/lib/main.py, shipped
via Dockerfile + entrypoint.sh + requirements.txt. The only PHP present is
phpcs-custom-sniffs/ (one sniff) plus analysis-bootstrap.php. Verified in
this repo: no phpunit.xml, no phpunit.xml.dist, no tests/, no pytest.ini,
no pyproject.toml, no conftest.py, no test_*.py, no *_test.py.

The composer.json nevertheless declared test:unit / test:all as

  if [ -f phpunit.xml ] || [ -f phpunit.xml.dist ]; then ./vendor/bin/phpunit
  ...; else echo "SKIPPED: no phpunit.xml in this repo ..."; fi

The condition is false by construction, so both scripts were a guaranteed
exit 0 that ran nothing - and check, check:full and check:strict all chained
them, so each aggregate carried a test leg that could never fail. It also
shipped test:coverage and coverage:check (a 75% clover threshold) for a
suite that does not exist.

A script that pretends to run a PHP test suite in a repo with no PHP
application code is worse than no script: it manufactures a green. Removing
it makes the absence of tests VISIBLE. check:strict now prints, on success,
exactly what it did and did not cover.

Removed: test:unit, test:all, test:coverage, coverage:check.
Rewritten: check, check:full, check:strict - no longer reference them.

Verified nothing else calls them:
- no reference in .github/workflows/
- the shared ConductionNL/.github quality.yml never invokes composer test:*
  (its frontend-tests job reads package.json, not composer.json), and
  enable-phpunit defaults to false and is not set by this repo, so no
  PHPUnit job existed in CI either.

Measured after the change (openklant, PHP 8.5 container):
  composer test:all   -> exit 1, "Command \"test:all\" is not defined"
  composer check:strict -> runs lint phpcs phpmd psalm phpstan only

NOTE: this repo still has NO automated tests of any kind. The Makefile
`test` target is `docker run --rm -it ...`, an interactive container launch
that asserts nothing and cannot run in CI. Wiring up a real pytest suite is
separate, larger work and is NOT done here.
@rubenvdlinde
rubenvdlinde merged commit 0e3da79 into development Aug 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant