Testing

Testing

Validation for Relay 0.3.0 preview, September 5, 2026. Disposable installations include a real WordPress 7.1 / PHP 8.3 / MariaDB 11.4 plugin lab plus separate targeted fixtures.

Executed checks

Environment	Result
WordPress 6.4.10 / PHP 7.4.33, standalone	27 integration checks plus 23 feature checks passed
WordPress 7.1 / PHP 8.4.23, standalone	27 integration checks plus 23 feature checks passed
WordPress 6.8.8 / PHP 8.3.32 / OpenStation 1.1.6, multisite	36 integration checks plus 23 feature checks passed
WordPress 6.8.8 / PHP 8.3.32 / OpenStation 1.1.6, single site	Native registration, data/action dispatch and browser interactions passed
Standard WordPress browser suite	Collection, recurring notices, archive/restore/snooze, search, warning/error/form/save-message visibility, failed saves, nonce rejection and reflow passed
Extended browser suite	Malformed payloads, unsafe markup/URLs, concurrent saves, anonymous denial, keyboard folder navigation, 150-record retention, HTTP failure, no-JS behavior, and 320/390/768/1440px widths passed
Native OpenStation browser suite	Real window mount, search and type filtering with no requests, archive/restore/snooze dispatch, local folders, desktop/mobile reflow, and no uncaught JS errors passed
Accessibility	Axe WCAG 2 A/AA and 2.1 AA scan: no detected violations within either Relay view
Package	Native app files included; tests and npm dependencies excluded; self-contained Playground blueprint boots
Real plugin lab	25 browser checks passed: actual plugin screens, WooCommerce draft save, genuine notice provenance, native and fallback interactions, live updates, URL routing, and closed-window dock badge
Live fallback and privacy	Six checks passed: localStorage-only invalidation, revision-only payloads, live snooze expiry, no idle request loops, different user scopes, and separate inbox records
Native settings	Five checks passed: custom datetime, retention confirmation, sender-rule preview/save/reset, settings accessibility, and confirmed history deletion including pins
Dependencies	npm audit: zero reported vulnerabilities after updating the test tool's transitive qs dependency

Automated accessibility checks are scoped to Relay, not the entire WordPress or OpenStation shell. They do not replace screen-reader testing. Native mobile checks used a 390px viewport. Browser tests used Chrome; Firefox, WebKit and real mobile hardware remain untested.

What the integration suite covers

Actual WordPress user metadata, current-user capability checks, input types, batch size, same-site URL reconstruction, archive persistence, 24-hour snooze and expiry, missing ids, duplicate notices, malformed records, markup stripping, overlapping and stale locks, database-write failures, lock release, separate administrators, subscribers, anonymous users, per-site isolation, and uninstall preserving unrelated metadata.

The native tests load the real OpenStation registry and runtime, not stubs. They check app discovery, companion-asset resolution, shared inbox data, mutation dispatch, trusted source lookup, and unauthorized-user denial.

The concurrent browser test accepts success or an explicit retryable conflict. Every successful write must remain in the subsequent record set; silently losing a successful write fails the test.

Findings fixed during testing

• A four-worker test exposed lost successful writes. Both UI routes now use an atomic shared write lease; the original add_option() lease was rejected because WordPress uses an upsert.
• A multisite environment check incorrectly hid the native app on ordinary sites: it now checks the actual network-admin screen.
• The native view used the wrong foreground token on dark themes: it now inherits OpenStation's palette.
• Secondary text in the WordPress view measured 4.43:1: it was darkened to meet the scan's required contrast.
• Malformed action/id payloads now return validation errors instead of reaching string processing.

Evidence and reproduction

Raw test evidence includes JSON integration reports, accessibility results, concurrent request statuses, and native/fallback screenshots. See Development for commands. The repository's GitHub Actions workflow runs the WordPress version matrix and standalone browser suite on pushes and pull requests. Native OpenStation checks are run locally against a built checkout; they are not yet part of hosted CI.

Remaining coverage

Broader third-party plugin workflows, network-admin UI behavior, Redis/Memcached-backed object caches, a killed process during an actual database write, screen-reader workflows, focus after every list mutation, translated admin locales, and Firefox/WebKit/mobile devices. Keep the release marked as a preview until the compatibility work is done.

Real plugin versions and exact scope

Plugin	Version	Executed interaction
WooCommerce	11.1.0	Entered a product title and price, saved a draft through the real product editor, and verified the title persisted.
Yoast SEO	28.4	Collected its actual first-time configuration notice with verified sender identity; exercised Relay read/pin/snooze/archive/rule controls on that notice; opened the Yoast admin screen.
Easy Updates Manager	9.0.22	Verified its real cron-configuration error remained visible; collected its sender identity; opened its settings screen.
WPForms Lite	2.0.1.1	Opened the actual forms overview with Relay active.
UpdraftPlus	1.26.7	Opened the actual backup screen with Relay active.
OpenStation	1.1.6, commit a48d37b8bfdc6453056eb9345ff610c396a71e15	Native mount, actions, custom snooze, settings, admin-link remap, live refresh, closed-app badge, and mobile reflow.

WPForms and UpdraftPlus coverage is a screen-load smoke test, not a completed form submission or backup/restore workflow. The late-arrival, changed-warning, failed-network, and nested-editor tests deliberately inject controlled fixtures; they are listed separately from genuine plugin notices. No production site or customer data was used.

The feature suite checks atomic invalid-batch rejection, Undo preserving newer choices, explicit unread counts, daylight-saving conversion, nonexistent local times, forged sender proofs, prospective rules, warning/error/pin exemptions, rule reset, retention, the 25-pin cap, and deletion confirmation. These use actual WordPress APIs, not mocked persistence.

A real-app regression caught an empty-ID content-change announcement that OpenStation silently ignores. Relay now announces one aggregate inbox document. Native rule choices use visible buttons after testing exposed a scroll-dismiss issue in the upstream select component inside the long settings panel.

Hosted CI

All four GitHub Actions jobs passed for commit 4574ec1068d60fcb6732134c6ce3ba97bc781ba9: three WordPress/PHP combinations and Linux Chromium browser, packaging, syntax, and dependency-audit checks. Native and real-plugin tests ran locally against the built upstream checkout described above.