Add the WordPress integration harness and the compatibility matrix - #5
Merged
Merged
Conversation
P0-0, and the machinery P0-1 needs. The unit suite stubs WordPress rather than loading it — right for a fast suite, useless for anything that has to exercise activation, a route, a capability or a cron event. Only two production files load under it, which is how a fatal on init survived a green CI for the whole life of the 2.0.0 release. Adds: - bin/install-wp-tests.sh — installs WordPress and the core test library. No svn: GitHub runners do not ship it, and a missing binary is a confusing way to fail a compatibility matrix. - phpunit-integration.xml — separate from phpunit.xml by necessity, not preference. The WordPress core test library still calls PHPUnit\Util\Test::parseTestMethodAnnotations(), which PHPUnit 10 removed, so every integration test errors under 10.x on 6.8, 6.9, 7.0 and trunk alike. The integration job installs 9.6 over the top; the unit suite keeps 10.5. Merge the two configs if WordPress ever adopts PHPUnit 10+. - tests/integration/ — bootstrap, base test case, record factory, and three suites: ActivationTest, FreshInstallTest, LoadDeprecationsTest. - .github/workflows/integration.yml — WordPress 6.8/6.9/7.0.3 x PHP 8.2/8.3/8.4, plus a non-blocking trunk canary. LoadDeprecationsTest is the part that finds version incompatibilities without anyone maintaining a per-release list of what was deprecated. WordPress reports its own; the bootstrap captures them across plugin load and activation, and the test asserts the set is empty. Its second test asserts the running WordPress matches the version the job installed — a matrix reporting green for a version it never exercised is exactly what `Tested up to` exists to prevent. Current stable WordPress determined at execution time rather than taken from documentation: 7.0.3, with 6.8.7 and 6.9.6 the other supported lines. Read from the WordPress/wordpress-develop tag list. Scope: the harness and P0-0's own acceptance tests. The REST authorization, ownership, route-inventory and webhook suites are P0-5 and P0-6 and get their own PRs; the route-inventory artifact step was removed from the workflow accordingly, so it does not reference a test that is not here. readme.txt `Tested up to` is deliberately NOT changed. Per P0-1 that moves only after the matrix passes, and the matrix has never run. Verified locally: php -l clean on every ported file; unit suite still 51 tests / 847 assertions / 0 failures; all three workflows parse as YAML; install-wp-tests.sh parses; the distributable still excludes tests/, bin/ and phpunit-integration.xml. NOT verified locally, and cannot be: the integration suite itself. This environment has no MySQL server and no Docker daemon to run one, and the gateway returns 403 for api.wordpress.org, so neither the database nor the WordPress download the harness needs is reachable. GitHub Actions is the first place this can execute — CI is the proof, and P0-0/P0-1 stay open until it is green. Also noted while here: composer.lock is already out of date with composer.json on main, independent of this branch. `composer install` warns rather than fails, so CI is unaffected. Not fixed here — regenerating the lock needs packagist, which this environment cannot reach. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LgGmwdEzTdVas4VTDzJQ5m
Shubochandrosarker
marked this pull request as ready for review
August 8, 2026 22:47
This was referenced Aug 8, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Before: nothing in this repository loads WordPress. The unit suite stubs it — right for a fast suite, useless for anything that must exercise activation, a route, a capability or a cron event. Only two production files load under it. After: a real WordPress and a real database, across every supported version.
That gap is not theoretical. It is exactly how the
initfatal fixed in #4 survived a green CI for the entire life of the 2.0.0 release.Delivers P0-0, and the machinery P0-1 needs.
Type
Rules touched
No runtime code changed. Not one file under
includes/,templates/orassets/is touched.Current stable WordPress, determined at execution time
Per §9, not taken from documentation — read from the
WordPress/wordpress-developtag list:Matrix: WordPress
6.8/6.9/7.0.3× PHP8.2/8.3/8.4, plus a non-blocking trunk canary. A green canary is never support for an unreleased version.What is here
bin/install-wp-tests.shphpunit-integration.xmltests/integration/bootstrap.php…/class-memberistic-integration-testcase.php,…-record-factory.phpActivationTest,FreshInstallTest,LoadDeprecationsTest.github/workflows/integration.ymlWhy two PHPUnit configs. Not preference — the WordPress core test library still calls
PHPUnit\Util\Test::parseTestMethodAnnotations(), which PHPUnit 10 removed. Every integration test errors under 10.x, identically on 6.8, 6.9, 7.0 and trunk, so no newer WordPress resolves it. The integration job installs 9.6 over the top; the unit suite keeps 10.5 and never loads WordPress. Merge the configs if WordPress adopts PHPUnit 10+.LoadDeprecationsTestis the load-bearing one. It finds version incompatibilities without anyone maintaining a per-release list of what got deprecated — a list that would be stale the day it was written. WordPress reports its own deprecations; the bootstrap captures them; the test asserts the set is empty. Its second test asserts the running WordPress matches the version the job installed — a matrix reporting green for a version it never exercised is precisely whatTested up toexists to prevent.Scope
The harness and P0-0's acceptance tests only.
RestAuthorizationTest,RestOwnershipTest,RestRouteInventoryTestandWebhookSecurityTestare P0-5/P0-6 and get their own PRs — the route-inventory artifact step was removed from the workflow so it doesn't reference a test that isn't here.readme.txtTested up tois deliberately unchanged. Per §13 and P0-1 it moves only after the matrix passes, and the matrix has never run.What I ran
I have not executed the integration suite. It cannot run in this environment:
/var/run/docker.sockabsentapi.wordpress.orgwordpress.org,downloads.wordpress.orgsvngithub.com(used to read the WP version)So GitHub Actions is the first place this code can execute, and this PR's CI run is its first real test rather than a confirmation. I expect to iterate on it. Treat a red integration job here as expected-and-being-fixed, not as a finished deliverable.
P0-0 and P0-1 stay open until that matrix is green. I have marked P0-0 in progress, not done.
Other gaps, stated plainly:
bin/install-wp-tests.shhas never been executed —bash -nonly.Also found
composer.lockis already out of date withcomposer.jsononmain, independent of this branch —git stash-free check againstmainreproduces it.composer installwarns rather than fails, so CI is unaffected. Not fixed here: regenerating the lock needs packagist, which this environment cannot reach (curl error 28, SSL connection timeout).Relatedly,
yoast/phpunit-polyfillsis not added tocomposer.json, though the reference implementation adds it. Adding it without being able to regenerate the lock would have made the mismatch worse and risked the unit-suite job. The integration workflow installs it explicitly at job time, so nothing needs it in the manifest.Backlog
Risk and rollback
tests/,bin/andphpunit-integration.xmlare all excluded, asserted above. The new workflow only adds a CI job.Generated by Claude Code