-
Notifications
You must be signed in to change notification settings - Fork 0
Reliability Milestone
Started September 4, 2026 to implement the Core-only feature recommendations and expand the reliability evidence. This page separates verified results from remaining launch gates. The Fleet feature work adds no managed-site endpoint.
- Publishing controls: featured media, categories/tags, author and discussion settings; preserve unsupported fields, include edits in conflict fingerprints and signed reviews.
- Draft crash recovery: explicitly enabled, encrypted hub recovery checkpoints, scoped to user/site/type/item/editor and connection generation. Uses Core Heartbeat/transients, not remote autosaves: Core's own-author draft autosave behavior can update the parent. Never silently create a remote post merely by opening an editor.
- Media uploads: bounded binary transport to Core, destination MIME/capability checks, no blind replay after uncertain creation.
- Comments: replies in context and reviewed bounded bulk moderation with per-item outcomes and no automatic retry of uncertain creates.
- Synchronization: last successful checks, stale/queued/backoff state, stalled cron diagnostics and recovery actions.
- Team access: explicit per-site hub grants, distinct read/editor/operator access, default private ownership, immediate server-side revocation and no credential exposure or arbitrary Explorer bypass.
- Mandatory CI: provision disposable real WordPress fixtures, test the installable ZIP, cold install/upgrade, credential/content flows, PHP and browser matrix.
- Stress/recovery: distinct-origin MySQL/MariaDB lab; final-build 30/50/100-site measurements; realistic large datasets; interrupted writes, concurrent writers, hostile inputs, permission changes, backup/restore and browser/cron lifecycle checks.
- Endurance: 48–72-hour opt-in soak with bounded load, counters and safe cleanup. A short verification run is not a completed soak.
Release status: 0.10.0 is the current GitHub release. The saved multi-instance defect is fixed and Fleet's dedicated regression passes with OpenStation PR #763, exact commit a48d37b8bfdc6453056eb9345ff610c396a71e15. That PR is still open and the fix is not part of a released OpenStation build. The failed 48-hour attempt has not been restarted or completed. Independent-hosting and agency pilots also remain open.
Implementation started from released commit 2057bc1 and wiki 59afc0e. Both working trees were clean. Tests and feature outcomes will be recorded here as they are verified, not inferred from the plan.
All six feature areas are implemented in the public 0.10.0 release: publishing controls and searchable name pickers, optional encrypted recovery, media upload, comment replies/reviewed bulk moderation, synchronization diagnostics and explicit team roles. No additional managed-site endpoint or Fleet-specific OpenStation API is required. The default remains one private connection per owner; sharing is deliberate.
The browser suite now includes real Core writes and native UI actions, not only mocked dispatch. New security cases cover recovery isolation/expiry ordering, malformed uploads before transport, concurrent content locks, shared-reader UI, grant revocation in an open window, bounded sharing indexes, metadata-only conflicts and uncertain reply/batch outcomes. The workflow adds a mandatory real-browser CI job on Chromium, Firefox and WebKit using the installable ZIP and an exact framework commit. Local execution remains separate evidence from hosted GitHub CI.
The larger lab caught two Fleet integration issues: the approval link must explicitly request Core's classic screen when the managed account has opted into OpenStation; and Fleet's browser bridge must wait for wp.os.ready() before using shell APIs. Both were corrected in Fleet using existing public integration points. An out-of-tree test plugin symlink also prevented framework stylesheet resolution; the lab now installs Fleet's actual ZIP under the hub's normal plugin directory. The separate saved-instance framework defect led to OpenStation PR #763.
- Final full browser matrix: 123/123 in Studio (8.8 minutes) and 123/123 in the MySQL/MariaDB lab (4.7 minutes), with zero configured local retries. Each run is 41 tests across Chromium, Firefox and WebKit. WebKit is not a physical Safari-device test. Its exact unsupported
interactive-widgetviewport warning is annotated; other console errors are not broadly suppressed. - Post-upgrade Chromium confirmation: 14/14 connection/reliability/security tests passed in 32 seconds. Independent cleanup inspection found zero run-owned credentials, zero injected outage flags, the temporary owner deleted and all 100 baseline connections preserved.
- PHP syntax, smoke/security checks, incremental search-index tests, the public-ready bootstrap regression, coding standards, static analysis, Composer validation/audit and npm audit passed. WordPress Plugin Check 2.1.0 reported no errors on the extracted runtime. The final source commit passed the expanded hosted workflow: PHP 8.3/8.5 builds, reproducibility, quality/audit, Plugin Check, a real 0.8.0 upgrade and clone restore, the Chromium/Firefox/WebKit mutation and isolation suite, and the saved multi-window regression. No flaky retry was accepted.
- Final reproducible runtime ZIP SHA-256:
af0f70a17af9cf8906cbb72f883a1e18502665639c1ac48863c5b5e97d1618d8. Source commit:6b7c8242fc160afeafd45a5ddcf92201777e100a. The version and release documentation changed after the fully tested candidate; application logic, JavaScript, and CSS did not. - Real upgrade: the pinned published 0.8.0 ZIP → the 0.10 ZIP preserved all 100 baseline encrypted connections, their generations and agency data. A Core authenticated read succeeded. A complete hub database export was imported into a separately named disposable clone; Fleet booted and authenticated from that clone while the original database remained unchanged. Missing/replaced salts failed closed and restored salts worked. The authoritative quiescent repeat completed at
2026-09-04T17:14:40Z; an earlier repeat overlapped the tail of load-test cleanup and is not the isolated upgrade evidence. Temporary clone/dump cleanup completed. This is not another provider's disaster-recovery certification. - All ten README captures were regenerated from the current runtime on September 5, with matching HTTP/local asset hashes. See native UI review.
The lab has 100 independent target databases/HTTPS origins with 1,000 seeded posts each, alternating MySQL 8.4/MariaDB 11.4 on WordPress 7.1/PHP 8.3.33. Core files, Apache/PHP workers and the machine/network are shared. Every load-test connection went through Core's real approval screen under a separate temporary hub owner, distinct from the baseline/soak owner.
| Connected sites | Hub open median / p95 | Cached model median / p95 | Three content windows | Overview remote requests |
|---|---|---|---|---|
| 30 | 392 / 435 ms | 2.14 / 2.91 ms | 1,018 ms | 0 |
| 50 | 382 / 387 ms | 2.09 / 2.56 ms | 920 ms | 0 |
| 100 | 404 / 421 ms | 3.81 / 4.60 ms | 664 ms | 0 |
These are warm local observations (five window opens and ten model samples per checkpoint), not production latency guarantees. Three site-content windows were opened sequentially and left open together. At 100, the hub model was 265,524 bytes and whole-request peak PHP allocation 68,485,120 bytes; neither figure isolates Fleet's own allocation.
Failure isolation passed: two targets unavailable, one load-only credential revoked, 97 healthy targets continued, and all three failures entered backoff and appeared in the hub. The three cron passes took 40.053, 40.030 and 29.823 seconds, within the 45-second test tolerance around the 40-second runtime budget. A pass may spend its budget on another owner's due work before continuing the next owner.
The run was resumed across fixes, not a pristine first-attempt pass. Earlier 10/20 checkpoints predated the final browser bootstrap patch and are excluded from the table. The 30/50/100 checkpoints used the final runtime. A same-host lab cookie accumulation problem stopped the first attempts at site 21: cookies are not port-scoped, unlike the origins. Keeping only the current target and unchanged hub session in the isolated browser allowed the actual 100 approvals to complete without changing production security limits. The lab also waits for public shell-ready/content-ready/close lifecycle boundaries. Sanitized local evidence: tests/stress/results/multi-origin-100/load-1788542032860.json. Cleanup reported no errors; credentials belonging to that run and its temporary owner were removed, while the 100 baseline connections were retained.
The three-cycle harness smoke passed: 300 real Core reads, six create/trash actions, three concurrently open windows per cycle, no reported page errors and about 10 MB sampled JS heap. The first smoke attempt correctly rejected an incomplete canary fixture before any remote post was created; the harness was corrected to submit all required editor fields and the exact target/title was independently checked for leftovers. This was a test-fixture correction, not a relaxed product validation rule.
The 48-hour attempt started at 2026-09-04T17:17:27Z and failed at 17:18:00Z, before completing its first cycle. Its 100 content reads passed; restoring/reopening windows did not. The process exited and the watcher was paused. There is no 48-hour coverage claim.
Follow-up isolation reproduced a framework defect, not simply a slow target: the saved fleet-site-2 instance was skipped at reload, then its staged restore parameters replaced the site explicitly requested for a new window. The regression performs no remote writes and cleans up its temporary owner. The soak harness now requires every saved native instance to restore before reusing an ID, and tests/lab/restore-regression.js is a mandatory CI gate.
OpenStation PR #763 preserves the saved instance ID through session restore and prevents staged parameters from being consumed by the wrong new window. With its exact head commit installed, the regression passed at 2026-09-04T19:06:42Z: fleet-site and fleet-site-2 restored their original sites, and a newly opened fleet-site-3 received the explicitly requested third site. This proves the focused identity case locally; the PR's review, checks, merge, and release remain upstream gates.
After the fix is available in a supported OpenStation package, rebuild the exact Fleet/OpenStation pair and deliberately restart the full-duration test from zero. Independent hosting trials and agency onboarding pilots also remain required.
The disposable Docker lab was stopped after verification to avoid idle resource use. Its databases/files are retained; the original Studio demo sites remain available. Rebuild/reprovision with the current lab configuration before future clone-restore testing: the upgrade harness now verifies the running configuration's exact hash before any database operation. The final guarded upgrade/clone check also passed at 2026-09-04T17:24:36Z.
Independent hosting-provider trials and three agency onboarding pilots require authorized staging hosts/participants. Do not provision paid hosting, contact agencies or claim certification without those inputs. The existing Core post update endpoint does not offer atomic compare-and-swap against unrelated external writers; hub-local serialization does not remove that remote race.