Skip to content

Load Testing

Jump to bottom
Nick Hamze edited this page Sep 5, 2026 · 2 revisions

Reproducing the 10 / 20 / 30-site test

This page preserves the original 0.9 Studio measurements. The newer 0.10 work adds distinct-origin MySQL/MariaDB 30/50/100-site fixtures, three-browser coverage, real ZIP upgrades/clone restores and a long-running soak. See Reliability milestone and the repository's tests/lab/README.md; do not attribute these historical timings to the new build.

These are real independent WordPress installations, not duplicated cards or mocked connection records. Each has its own SQLite database, administrator, Core REST surface, and Application Password. They share a local HTTPS origin and one Studio PHP server; the hub has a separate server. This measures local integration/scaling, not distributed hosting latency or MySQL performance.

Disposable lab

Create a new Studio site named Fleet Launch Load Lab, path ending in /fleet-launch-load, with local HTTPS. Start the existing test hub separately. Never use production paths.

From Fleet's repository:

FLEET_STRESS_WRITES=1 \
FLEET_STRESS_ROOT=/absolute/Studio/fleet-launch-load \
FLEET_STRESS_URL=https://fleet-launch-load.wp.local \
FLEET_STRESS_OPENSTATION=/absolute/compatible/desktop-mode \
node tests/stress/provision.js

FLEET_STRESS_WRITES=1 \
FLEET_STRESS_ROOT=/absolute/Studio/fleet-launch-load \
FLEET_E2E_HUB_PATH=/absolute/Studio/fleet-hub \
node tests/stress/run.js

The hub's explicitly local test adapter must permit these private HTTPS fixtures and their development certificate. Production Fleet retains safe-HTTP and TLS protections. The provisioner supplies fixture-only Studio URL/proxy adapters and a controlled 503 fault switch. None ships in the ZIP. Core files are independent copies, not symlinked to a shared config/database.

What is exercised

  • Native WordPress approval and explicit setup for 30 real connections, owned by a temporary hub administrator.
  • At 10, 20 and 30: unique connection count; completed setup; ten cached-model samples; five actual hub opens; three simultaneously open site-content views checked against their own records.
  • Cached hub rendering must make zero remote HTTP requests.
  • At 30: two targets return 503 and one credential is revoked. Twenty-seven healthy sites must continue checking; three failures must enter backoff and appear in the hub.
  • Scheduled passes must stay within a 45-second test tolerance around Fleet's 40-second budget.
  • Cleanup revokes only the run's credentials and deletes only its temporary hub user/connection data. Fixtures remain for reproducibility and should be stopped after testing.

Artifacts and recovery

tests/stress/results/ is ignored by Git. Reports contain counts, timing, sizes, versions, fault outcomes, and cleanup status—not credentials, cookies, authorization headers, or callback URLs. Browser traces/screenshots are disabled during approval.

A failed run keeps a non-secret ownership journal so the same command can resume. To revoke/remove its temporary connections without running the browser, add FLEET_STRESS_CLEANUP=1 to the run command. Do not delete the journal before cleanup. A successful run cleans credentials automatically.

Model timings exclude WordPress bootstrap; browser timings include window closing/opening/rendering. PHP peak memory covers the whole bootstrapped request, not Fleet alone. These small-sample figures are observations, not an SLA. Public launch still requires different hosting providers, network failures, and the pilot in Launch checklist.

Recorded results

The figures below remain valid for their measured paths. A subsequent edge-case pass found four unrelated or more narrowly scoped bugs, including lost unsaved text and stale health handling. Passing this load scenario does not mean the candidate is bug-free or launch-ready.

Completed September 4, 2026, using Fleet 0.9.0-rc.1, WordPress 7.1, Studio 1.20.0, native PHP 8.3.33 servers (four target workers and a separate hub pool), and WP-CLI PHP 8.5.10. OpenStation was built from commit 9bac9176b26a6228e3e57c9a7c8f11f586063afd.

Connected sites Hub open median Hub open p95 Cached model median / p95 Three site-content windows Overview remote requests
10 468 ms 1,025 ms 0.22 / 0.37 ms 4,707 ms 0
20 291 ms 904 ms 0.41 / 0.53 ms 4,641 ms 0
30 292 ms 300 ms 0.63 / 0.74 ms 3,856 ms 0

The serialized hub model grew from 15,474 to 43,690 bytes; whole-request PHP peak memory stayed at 70,582,272 bytes (about 67.3 MiB). Browser timings include five opens, not enough samples for a production percentile claim. Later stages benefited from warm caches. Three content windows were opened sequentially and remained open together; the time is their combined load time, not a single-request latency.

Failure isolation passed: two fixtures returned HTTP 503 and one dedicated Application Password was revoked. All 27 remaining connections completed status checks, all three failures entered backoff, and the hub displayed three connection errors. The successful scheduled pass took 17.717 seconds, below the 40-second runtime budget. No browser page errors were recorded by the load harness.

What the test found and changed

The first failure test caught revoked credentials appearing healthy: WordPress's public REST index still succeeded while private collection requests failed. Fleet now checks the current authenticated administrator explicitly, retains the last inbox on failed verification, and backs off. An intermediate implementation also exposed two Core details: /users/me does not opt into batch requests, and its dynamically keyed capability map must be requested as a whole field. Both were corrected and the same 30-connection session passed on rerun. Regression checks cover revoked/forbidden/lost-capability accounts and recovery.

The table records the successful 10/20/30 connection checkpoints before this final authentication fix; the failure phase was resumed after it. They are separate measured phases, not a claim that an untouched first run passed. Early harness failures also led to waiting for actual content-ready/action-complete state, close-animation DOM removal, and shadow-DOM table rows. No credential-bearing traces were collected.

Cleanup was independently verified: zero remaining lab Application Passwords, zero load-test users, and the original demo user's two connections preserved. Only the new load server was stopped; fixtures remain on disk for reproduction.

Candidate verification

  • Full final browser suite: 10 passed in 58.9 seconds, including native approval/setup/repair/disconnect, revoked-credential detection, posts/pages source round trips, stale-edit protection, duplicate-create suppression, independent windows, control alignment, redacted local diagnostics, and axe WCAG A/AA checks.
  • PHP coding standards/static analysis, smoke checks, incremental index tests, and WordPress Plugin Check passed.
  • Reproducible runtime ZIP SHA-256: 34044d17d21ea827d3a8318ba62688629f12171edb30cbc36d9bff94403d6eef.
  • Source and wiki changes were local at the time of this load test. See the release record for later publication; these measurements are not a new run against that final artifact.

Raw sanitized run evidence: September 4 load run. Runtime screenshots and visual limitations: Taste review.

Clone this wiki locally