Skip to content

Add the WordPress integration harness and the compatibility matrix - #5

Merged
Shubochandrosarker merged 1 commit into
mainfrom
fix/current-wordpress-compatibility
Aug 8, 2026
Merged

Add the WordPress integration harness and the compatibility matrix#5
Shubochandrosarker merged 1 commit into
mainfrom
fix/current-wordpress-compatibility

Conversation

@Shubochandrosarker

Copy link
Copy Markdown
Contributor

What this changes

Before: nothing in this repository loads WordPress. The unit suite stubs it — right for a fast suite, useless for anything that must exercise activation, a route, a capability or a cron event. Only two production files load under it. After: a real WordPress and a real database, across every supported version.

That gap is not theoretical. It is exactly how the init fatal fixed in #4 survived a green CI for the entire life of the 2.0.0 release.

Delivers P0-0, and the machinery P0-1 needs.

Type

  • Release/CI plumbing
  • Feature (test infrastructure)

Rules touched

  • None of the above

No runtime code changed. Not one file under includes/, templates/ or assets/ is touched.

Current stable WordPress, determined at execution time

Per §9, not taken from documentation — read from the WordPress/wordpress-develop tag list:

Line Latest
6.8 6.8.7
6.9 6.9.6
7.0 7.0.3 ← current stable

Matrix: WordPress 6.8 / 6.9 / 7.0.3 × PHP 8.2 / 8.3 / 8.4, plus a non-blocking trunk canary. A green canary is never support for an unreleased version.

What is here

File Why
bin/install-wp-tests.sh Installs WordPress + the core test library. No svn — GitHub runners don't ship it, and a missing binary is a confusing way to fail a compatibility matrix.
phpunit-integration.xml Separate config, by necessity — see below
tests/integration/bootstrap.php Loads WordPress, captures deprecations across load and activation
…/class-memberistic-integration-testcase.php, …-record-factory.php Base case and fixtures
ActivationTest, FreshInstallTest, LoadDeprecationsTest P0-0's own acceptance: schema, DB version, capabilities, roles, safe defaults
.github/workflows/integration.yml The matrix + canary

Why two PHPUnit configs. Not preference — the WordPress core test library still calls PHPUnit\Util\Test::parseTestMethodAnnotations(), which PHPUnit 10 removed. Every integration test errors under 10.x, identically on 6.8, 6.9, 7.0 and trunk, so no newer WordPress resolves it. The integration job installs 9.6 over the top; the unit suite keeps 10.5 and never loads WordPress. Merge the configs if WordPress adopts PHPUnit 10+.

LoadDeprecationsTest is the load-bearing one. It finds version incompatibilities without anyone maintaining a per-release list of what got deprecated — a list that would be stale the day it was written. WordPress reports its own deprecations; the bootstrap captures them; the test asserts the set is empty. Its second test asserts the running WordPress matches the version the job installed — a matrix reporting green for a version it never exercised is precisely what Tested up to exists to prevent.

Scope

The harness and P0-0's acceptance tests only. RestAuthorizationTest, RestOwnershipTest, RestRouteInventoryTest and WebhookSecurityTest are P0-5/P0-6 and get their own PRs — the route-inventory artifact step was removed from the workflow so it doesn't reference a test that isn't here.

readme.txt Tested up to is deliberately unchanged. Per §13 and P0-1 it moves only after the matrix passes, and the matrix has never run.

What I ran

$ php -l on every ported file                    clean
$ vendor/bin/phpunit -c phpunit.xml              OK (51 tests, 847 assertions)
$ yaml.safe_load ci.yml release.yml integration.yml    all three valid
$ bash -n bin/install-wp-tests.sh                parses
$ bin/build-dist.sh && bin/assert-dist-clean.sh  distributable is clean
    correctly excluded: phpunit-integration.xml
    correctly excluded: tests
    correctly excluded: bin

⚠️ What I could NOT run — and why CI is the proof

I have not executed the integration suite. It cannot run in this environment:

Requirement Status here
MySQL/MariaDB server ✗ not installed
Docker daemon (to run one) ✗ CLI present, no daemon/var/run/docker.sock absent
api.wordpress.org gateway returns 403 — policy denial, confirmed via the proxy status endpoint
wordpress.org, downloads.wordpress.org ✗ unreachable
svn ✗ not installed
github.com (used to read the WP version) ✓ reachable

So GitHub Actions is the first place this code can execute, and this PR's CI run is its first real test rather than a confirmation. I expect to iterate on it. Treat a red integration job here as expected-and-being-fixed, not as a finished deliverable.

P0-0 and P0-1 stay open until that matrix is green. I have marked P0-0 in progress, not done.

Other gaps, stated plainly:

  • The three integration tests have never asserted anything against a real WordPress. They are ported and lint-clean; that is all I can currently claim.
  • bin/install-wp-tests.sh has never been executedbash -n only.
  • The trunk canary is untested like everything else here.

Also found

composer.lock is already out of date with composer.json on main, independent of this branch — git stash-free check against main reproduces it. composer install warns rather than fails, so CI is unaffected. Not fixed here: regenerating the lock needs packagist, which this environment cannot reach (curl error 28, SSL connection timeout).

Relatedly, yoast/phpunit-polyfills is not added to composer.json, though the reference implementation adds it. Adding it without being able to regenerate the lock would have made the mismatch worse and risked the unit-suite job. The integration workflow installs it explicitly at job time, so nothing needs it in the manifest.

Backlog

Risk and rollback

  • Blast radius: none at runtime. No shipped file changes; the distributable is byte-identical apart from nothing at all — tests/, bin/ and phpunit-integration.xml are all excluded, asserted above. The new workflow only adds a CI job.
  • Rollback: revert the commit; deletes a workflow and a test directory.
  • Data migration to reverse? No.

Generated by Claude Code

P0-0, and the machinery P0-1 needs. The unit suite stubs WordPress rather
than loading it — right for a fast suite, useless for anything that has to
exercise activation, a route, a capability or a cron event. Only two
production files load under it, which is how a fatal on init survived a green
CI for the whole life of the 2.0.0 release.

Adds:
- bin/install-wp-tests.sh — installs WordPress and the core test library. No
  svn: GitHub runners do not ship it, and a missing binary is a confusing way
  to fail a compatibility matrix.
- phpunit-integration.xml — separate from phpunit.xml by necessity, not
  preference. The WordPress core test library still calls
  PHPUnit\Util\Test::parseTestMethodAnnotations(), which PHPUnit 10 removed,
  so every integration test errors under 10.x on 6.8, 6.9, 7.0 and trunk
  alike. The integration job installs 9.6 over the top; the unit suite keeps
  10.5. Merge the two configs if WordPress ever adopts PHPUnit 10+.
- tests/integration/ — bootstrap, base test case, record factory, and three
  suites: ActivationTest, FreshInstallTest, LoadDeprecationsTest.
- .github/workflows/integration.yml — WordPress 6.8/6.9/7.0.3 x PHP
  8.2/8.3/8.4, plus a non-blocking trunk canary.

LoadDeprecationsTest is the part that finds version incompatibilities without
anyone maintaining a per-release list of what was deprecated. WordPress
reports its own; the bootstrap captures them across plugin load and
activation, and the test asserts the set is empty. Its second test asserts the
running WordPress matches the version the job installed — a matrix reporting
green for a version it never exercised is exactly what `Tested up to` exists
to prevent.

Current stable WordPress determined at execution time rather than taken from
documentation: 7.0.3, with 6.8.7 and 6.9.6 the other supported lines. Read
from the WordPress/wordpress-develop tag list.

Scope: the harness and P0-0's own acceptance tests. The REST authorization,
ownership, route-inventory and webhook suites are P0-5 and P0-6 and get their
own PRs; the route-inventory artifact step was removed from the workflow
accordingly, so it does not reference a test that is not here.

readme.txt `Tested up to` is deliberately NOT changed. Per P0-1 that moves
only after the matrix passes, and the matrix has never run.

Verified locally: php -l clean on every ported file; unit suite still 51 tests
/ 847 assertions / 0 failures; all three workflows parse as YAML;
install-wp-tests.sh parses; the distributable still excludes tests/, bin/ and
phpunit-integration.xml.

NOT verified locally, and cannot be: the integration suite itself. This
environment has no MySQL server and no Docker daemon to run one, and the
gateway returns 403 for api.wordpress.org, so neither the database nor the
WordPress download the harness needs is reachable. GitHub Actions is the first
place this can execute — CI is the proof, and P0-0/P0-1 stay open until it is
green.

Also noted while here: composer.lock is already out of date with composer.json
on main, independent of this branch. `composer install` warns rather than
fails, so CI is unaffected. Not fixed here — regenerating the lock needs
packagist, which this environment cannot reach.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LgGmwdEzTdVas4VTDzJQ5m
@Shubochandrosarker
Shubochandrosarker marked this pull request as ready for review August 8, 2026 22:47
@Shubochandrosarker
Shubochandrosarker merged commit 93190c0 into main Aug 8, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants