bench(db): Turso cluster per-vhost matrix + v0.8.x pin sweep + repro fixes - #4
Merged
Merged
Conversation
DB-BENCH.md's historical banner promised that a future pin bump would
replace the removed rusqlite/sqld lanes with a Turso-single vs
Turso-CDC-clustered matrix, added alongside rather than by editing them.
This is that suite, plus one axis the promise predates: per-VHOST
clustered replication (ephpm#416), where ownership of a site is decided
by rendezvous hashing and a non-owner forwards every ephpm_db_* statement
to the owner over sql/<site>.
Five lanes, one vhost, `bridge-point` / `bridge-write` cells at c=1 and
c=16 with the same warmup-plus-two-reps shape as every other suite:
S-turso-single no cluster; the anchor and the tie-back to the
historical `bridge` suite's B-turso lane
W-cluster-primary whole-DB clustered, measured on the primary
P1-persite-single one DB per vhost, single node (v0.7.0+ default)
P2-persite-owner one REPLICATED DB per vhost, on the site's owner
P3-persite-remote the same cluster and site, on a non-owner
P2 -> P3 is the forward hop. The three multi-tenant lanes also measure a
`wire-point` cell over stock pdo_mysql, which is NOT forwarded, so it
makes the hop claim falsifiable instead of asserted: if P3's wire cell
were also slower, the cause would not be forwarding.
parse.sh needed no changes -- `bridge-point`, `bridge-write` and
`wire-point` were already in its suffix list.
The mode gate earned its place immediately. Four database modes are
selected by a conjunction of keys across four config sections, and
ephpm-config does not reject unknown fields. The first run of the
per-site clustered lanes against the newest published image produced a
healthy, fast, fully-2xx three-node cluster running whole-DATABASE
clustered mode, because `per_site` does not exist in that image. Every
other gate passed. Each lane now asserts a specific startup line before
measuring anything, and P2/P3 refuse to run with a message naming
EPHPM_PERSITE_CLUSTER_IMAGE rather than reporting a mislabelled result.
Two further gates specific to this suite: a negative control (the same
fixtures without the vhost Host header must fail with "no per-site
database context", proving the tenant comes from the request and not the
mount), and an ownership gate requiring exactly one node to be named
owner by two independent observations (ephpm_cdc_subscribers on /metrics
and `elected as SQLite primary` in the log). /_ephpm/primary cannot serve
here -- in per-site mode it answers 200 on every healthy node by design.
The convergence proof writes through the bridge and counts over
pdo_mysql, and the direction is load-bearing: a bridge-side count on a
non-owner forwards to the owner, so it would agree with itself even if
replication were dead.
Setup ordering is documented because it has teeth: on a non-owner the
bridge hands back a remote proxy and never opens the site's local file,
so the open-hook never fires and that node never starts a replica driver.
A stock pdo_mysql request is what opens it. A per-site clustered
deployment whose apps use only the drop-ins may never open a tenant's
database on the nodes that do not own it.
The divergence probe (stock pdo_mysql writes are not forwarded) runs
AFTER the measurements, never before -- the same ordering rule the pooled
proxy lanes learned the hard way when a pre-measurement probe poisoned
the pool and produced 876 req/s of pure HTTP 500.
The driver now picks a default image PER SUITE: the five pre-v0.7.0
suites keep v0.6.3 (bumping them would replace a measurement with a
failed launch -- `engine = "sqlite"` is a hard startup error on v0.7.0+),
and `cluster` tracks the newest published image.
Status: authored and gate-validated on ephpm/ephpm:v0.8.5-php8.5, NOT
recorded. Lanes P2/P3 have never run in their intended mode -- per-site
clustered replication first appears in the unreleased v0.8.6 tag. No
reference numbers are published.
Every ePHPm pin in k8s/ was v0.6.3-php8.4, three minor lines behind the newest published image. All twelve now pin v0.8.5-php8.4. This is safe to bump and the db/ tier is not: I checked every ePHPm config the manifests actually use -- inline ConfigMaps, start-script heredocs, and the wordpress-v5/ scripts -- and none of them sets a knob v0.7.0 removed. The blocker is entirely inside db/configs/, where fourteen files set `engine = "sqlite"` and eight sweep `write_permits`. Those stay pinned to v0.6.3 as the historical record. The pin moved; the recorded numbers did not, and the tables say so rather than being quietly restated against the new pin. README and RUNTIMES-BENCH now carry an explicit note that their rows were measured on v0.4.0-v0.6.3 and are due for a re-record -- the repo's own rule is that a version bump changes the effective configuration (v0.5.0 autotuning being the documented example) so numbers recorded across a bump are not comparable. Also corrected, all version claims that had drifted into the present tense: - OPCACHE-CLUSTER.md claimed the manifests pin v0.5.0. They pinned v0.6.3 even before this change -- the doc was already a bump behind its own manifest. It now separates the FEATURE floor (>= v0.4.0) from the pin. - README's first sentence named v0.5.0 and "the current worker retest on v0.6.0", neither of which appeared in any manifest. - README's roadmap asked for an Octane/Swoole/RoadRunner/ePHPm comparison that RUNTIMES-BENCH.md already contains. - RUNTIMES-BENCH's "Experimental: Turso engine db lane" described `engine = "turso"` as experimental and warned the lane would "collapse on any future v0.7.0+ pin bump". That bump is this commit: the knob is now a no-op restatement of the default, so the lane duplicates bench-ephpm rather than contrasting with it. The lane is kept but relabelled, and the comparison it was reaching for is pointed at the new `cluster` suite -- on the local tier, where a microsecond-wide effect is actually resolvable. - The db-engines row in README's benchmark map is marked historical, and a row for the new `cluster` suite added, marked authored-not-recorded.
Found while sweeping the repo for the pin bump. Each of these turns a silent failure into either a working command or an actionable message. bench-proxy.sh: lanes D-I and the pg-cliff probe need containers named dbbench-mysql and dbbench-pg, which no script in the repo creates. That was an unstated assumption, so a first-time user got six lanes of gate failures with no indication why -- including the lanes behind the published "postgres +117%" number, which is the first thing a skeptical reader will try to reproduce. The upstreams are deliberately NOT started per-run (a fresh mysql:8 would add minutes of initdb to every invocation); instead the script now checks for them and prints the exact two podman commands. Its gate list also referenced a parse-proxy.sh that does not exist; the parser is parse.sh. probe-pg.sh and probe-clean-vs-dirty.sh never created their results directory, so on a fresh clone (db/results-*/ is gitignored) both died on a redirect. All three probes also used --network dbbench-net without creating it, working only as a side effect of having run a bench script first -- undocumented, since DB-BENCH.md lists them as standalone. render-laravel-v4.sh substituted a placeholder token that no longer exists in laravel-v4.yaml, while still refusing to run without EPHPM_SOURCE_IMAGE: you had to supply a value that was then discarded, and run-v4-worker-baseline.sh (which reads .generated/) could not proceed without inventing one. It now rewrites the real image lines when the variable is set and passes the manifest through when it is not. scale/README.md pointed at a report/ directory that is named reports/ -- including a `cd report && go run .` that cannot work. Its own provenance note already said the directory was renamed on import. New db/cleanup.sh: the bench scripts reclaim volumes at the START of a run, which is right for re-running a lane but means everything outlives it. `run-db-bench.sh all` leaves roughly ten named volumes -- one holding a full seeded WordPress tree -- plus a network, and no script removes any of it. The names span five prefixes while sharing one network, so the obvious `podman rm -f dbbench-*` misses half. Dry-run by default; --yes to remove. It never touches db/results-*/: that is the measurement, and a cleanup script must not be able to delete a result. (bench-cluster.sh already cleans up after itself; the older suites are left alone because their traps are load-bearing for the historical lanes.)
This was referenced Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Modernizes the lab for the v0.8.x line and delivers the suite
DB-BENCH.mdpromised when it marked the
engines/admissionlanes historical:Added alongside those lanes, not by editing them. The historical suites are
untouched and stay pinned to v0.6.3.
The new
clustersuiteFive lanes, one vhost, on the local podman tier (one host,
--cpus 1pernode,
oha, warmup + two timed reps) — the same shape as every other suitein
DB-BENCH.md. Cells arebridge-point(10 sequential point SELECTs viaephpm_db_query()) andbridge-write(1 INSERT viaephpm_db_execute()) atc=1 and c=16.
S-turso-singlebridgesuite's B-turso laneW-cluster-primaryS→W= "what does clustering cost"P1-persite-singleP2-persite-ownerP1→P2= "what does clustering cost a tenant"P3-persite-remotesql/<site>forward hop.P2→P3is the headlineThe three multi-tenant lanes add a
wire-pointcell over stockpdo_mysql.That is not a duplicate:
pdo_mysqlis not forwarded to the owner (thedocumented ephpm#416 gap), so on a non-owner the bridge cell pays the hop and
the wire cell does not. Measuring both makes "the difference is the hop"
falsifiable rather than asserted.
parse.shneeded no changes —bridge-point,bridge-writeandwire-pointwere already in its suffix list.
Validation status — please read before merging
Authored and gate-validated. Not recorded. No reference numbers published.
ephpm/ephpm:v0.8.5-php8.5onpodman: mode gates pass, fixtures return the canonical
sum:55, whole-DBreplication converged in ~1 s, every cell 100 % 2xx,
parse.shrenders them.Those were short runs to prove the lanes produce clean numbers — they are
not measurements and none of them appear in the docs.
replication (ephpm#416) is not in any published image: it first appears in
the v0.8.6 tag, whose Docker images are not on Docker Hub yet. Point
EPHPM_PERSITE_CLUSTER_IMAGEat a v0.8.6+ build to run them.The mode gate earned its place on the first run
Pointed at the newest published image, the per-site clustered lanes produced a
healthy, fast, fully-2xx three-node cluster. It was running whole-database
clustered mode, because
per_sitedoes not exist in that image andephpm-configdoes not reject unknown fields. Every gate except the logassertion passed. Without it this PR would have shipped a per-vhost forward-hop
number measured on a topology that has no forward hop.
Each lane now asserts a specific startup line before measuring, and P2/P3
refuse with a message naming the override rather than reporting a mislabelled
result. Two further gates: a negative control (same fixtures without the
vhost
Hostheader must fail with "no per-site database context", proving thetenant comes from the request, not the mount) and an ownership gate
requiring exactly one node named owner by two independent observations
(
ephpm_cdc_subscriberson/metricsandelected as SQLite primaryin thelog).
/_ephpm/primarycannot serve here — in per-site mode it answers 200 onevery healthy node by design.
An upstream finding worth a look
On a non-owner the bridge hands back a remote proxy and never opens the
site's local database file, so the registry's open-hook never fires and that
node never starts a replica driver for the site — it replicates nothing. A
stock
pdo_mysqlrequest is what opens it locally. The harness therefore hitscount.phpon every node before gating on convergence, and the ordering isdocumented. Outside the lab this means a per-site clustered deployment whose
apps use only the
ephpm_db_*drop-ins may never open a tenant's database onthe nodes that do not own it. Flagging it here rather than filing upstream.
Pin bump sweep
k8s/→v0.8.5-php8.4. I checked every ePHPmconfig those manifests use — inline ConfigMaps, start-script heredocs,
wordpress-v5/scripts,scale/config/ephpm.tmpl.toml,kv/andcontainment/heredocs — and none sets a knob v0.7.0 removed.db/deliberately not bumped. 14 configs setengine = "sqlite"(a hardstartup error on v0.7.0+) and 8 sweep the removed
write_permits. Bumpingwould replace a measurement with a failed launch.
run-db-bench.shnow picksa default image per suite: v0.6.3 for the five historical suites, newest
published for
cluster.say which version each table was measured on and that they are due for a
re-record, per the repo's own rule about comparing across a bump.
Other things found while sweeping
bench-proxy.shnever creates its upstreams. Lanes D–I need containersnamed
dbbench-mysql/dbbench-pgthat no script in the repo starts — so sixlanes, including the ones behind the published "postgres +117 %" number, fail
their gates with no explanation. Now checked with the exact two commands
printed. (Not auto-started: a fresh
mysql:8per run adds minutes of initdb.)bench-proxy.shcited aparse-proxy.shthat does not exist.redirect on a fresh clone; all three assumed a network a bench script had to
have created first.
render-laravel-v4.shsubstituted a token that no longer exists while stillrefusing to run without
EPHPM_SOURCE_IMAGE— blocking a documentedreproduction path behind a value it then discarded.
scale/README.mdpointed atreport/(actual:reports/), including acd report && go run .that cannot work.OPCACHE-CLUSTER.mdclaimed a v0.5.0 pin; the manifest already said v0.6.3.asked for a runtimes comparison the repo already ships.
db/cleanup.sh— the suites reclaim volumes at the start of a run,so
run-db-bench.sh allleaves ~10 named volumes (one holding a seededWordPress tree) plus a network, across five name prefixes. Dry-run by
default. It never touches
db/results-*/.Not addressed
measurement run, not a code change.
where the interesting failure modes live and this suite deliberately does not
go there.
RUNTIMES-BENCH.md's Class A rows are still v0.4.0 numbers under a v0.8.5pin. Labelled, not fixed.