v0.11.0

The release that measured first, then ran what it had written down.

The knobs shipped in the previous release were rebuilt on the recommended values and
re-measured. Workers double up to 4, then give about 10%. A pool 30× larger changes
nothing. Exactly one knob made a difference, and it was invisible in rps —
counting the round trips is what made it decisive.

Along the way: a restore procedure written that same day did not start, and
/readyz probed a database the resolver does not read. Both pass every check that
exists. This is the release that found what only running things finds — and then made the
running into a check: an end-to-end suite that builds the production shape and drives
it over HTTP.

Added

• An end-to-end check (tests/e2e/, uv run pytest -m e2e). It builds the
  production shape and drives it over plain HTTP: PostgreSQL in docker, a minter and
  a resolver started separately under uvicorn, the ledger built through the CLI,
  and minting and resolution done with the key that CLI printed.

  The rest of the suite calls the app directly with TestClient, on SQLite, with
  authentication substituted — faster, and a finer net for authorisation. What this
  catches is what passes through that net: that the app is a factory (arkhe.app:app
  does not start), that the minter has no resolution route and the resolver has no
  minting route, and that the resolver never touches the write database.

  The resolver's write URL points nowhere. Touch it and the check fails — reverting
  this release's /readyz fix does exactly that, which was verified.

  It is step 5 of check.sh, and prints SKIP when docker is missing rather than
  passing quietly.

• Figures re-measured on the recommended settings, in
  Deployment. A ledger of one million
  ARKs, four resolver workers, POOL_SIZE 3 and MAX_OVERFLOW 2: resolution reaches 938
  rps at concurrency 8, p50 6.5 ms.

  Three things came out of it. Throughput flattens at concurrency 8 and beyond that only
  latency grows. Only 16 connections are ever open (20 available, 179 still free of
  max_connections=200) — the recommended pool is not too tight. And minting's 59.7 ms
  is almost entirely Argon2, which is why bulk is 55 times faster: one
  authentication divided across a thousand rows, not a faster database.

• How it scales out, and what jams first, measured and written up in
  Deployment.

  Workers double up to four (352 → 718 → 1,413 rps) and buy about ten per cent after
  that. A thirtyfold sweep of the pool moves the rps not at all — it moves only how
  many connections are held, because resolution is one short query with nothing to hoard.
  What jams next is PostgreSQL's CPU (roughly 1.4 : 1 against the app).

  A section on proving you are measuring the server comes first. bench.py tops out
  near 2,200 rps per process (the GIL), so anything above that describes the client —
  four processes in parallel reach 9,017 rps.

  It also records that repeating the same configuration varies by ±20%, and that a
  difference smaller than that band is not a difference.

• ARKHE_DB_PRE_PING — the liveness check on a pooled connection can now be turned
  off (it stays on by default).

  Turning it off roughly halves the database round trips per resolution: measured
  1.60 → 0.82 (counting xact_commit + xact_rollback against a million-ARK ledger).
  A resolution reads one index, so the extra verification round trip is heavy in
  relative terms.

  The rps never showed it — it drowned in the noise. Counting made it decisive.

• A sweep of the database and application knobs, and what did nothing, in
  Deployment. uvloop /
  httptools, shared_buffers at 128MB / 512MB / 2GB, a thirtyfold pool sweep,
  synchronous_commit=off for minting — none of them moved the number.

  The default 128MB of shared_buffers already gives 99.95% cache hits. A resolution
  touches one index and one row, so the general "25% of RAM" buys nothing here. Same
  for synchronous_commit: 53 of the 60 ms a mint takes is Argon2 — if giving up
  durability buys nothing, there is no reason to give it up.

• What a replica actually does, measured and written up in
  Deployment.

  The rps does not go up (1,433 → 1,307 — on one host a replica adds no CPU). What
  grows is the primary's headroom: the read load leaves it entirely (136% → 0% CPU). A
  replica buys not speed but minting not being dragged down when reads get rough.
  Resolution also survives the primary being stopped (302, ?info still 200).

  Replication lag from minting to being resolvable was 3–5 ms median (replay_lag
  1.3 ms even under write load), and all 50 attempts resolved on the very first
  request — the 404 warned about in the monitoring section never once happened under
  these conditions. The warning stays, with the conditions that would produce it spelled
  out (another machine, a replica busy with reads, a large ingest).

• Three procedural notes for following a procedure (.claude/skills/):
  release (the six steps of cutting a version), measure (how to measure), chaos
  (breaking it on purpose).

  What they record is mistakes. Appending a version section to an already-released
  one, measuring the client instead of the server, re-applying an expired chaos object
  and reading "no effect" — each happened. AGENTS.md is the source of truth; these
  are notes for carrying it out.

Changed

• The step for whoever embeds arkhe was removed from AGENTS.md. Advancing a
  submodule pointer is the embedder's procedure, and this document is arkhe's
  guide. Written in two places, one of them goes stale.

Fixed

• The start-up command in the restore procedure did not work. It said
  uvicorn arkhe.app:app, but the app is a factory (arkhe.app:create_app --factory)
  and will not start that way — found by actually running the procedure written moments
  earlier. Everywhere else (Quickstart, the federation guide, the Dockerfile, compose)
  was right.

• scripts/bench.py sent fewer requests than -n. Integer division by the concurrency
  meant -n 3000 -c 16 sent 2992. Compared against -n, the status breakdown looks like
  eight failures — which is exactly how it was misread. The remainder is now distributed,
  so what it counts matches what it said it would send.

• /readyz probed a database the role does not read. A resolver reads from
  ARKHE_READ_DATABASE_URL (the replica), yet the probe went to the primary — so
  with the replica down and resolution returning 500, it kept answering Ready as
  long as the primary was alive, and a load balancer kept sending traffic.

  Found by actually stopping the replica. It now probes the side the role reads (the
  read connection for a resolver, the write one for a minter).

---

日本語の変更履歴
· Full Changelog: v0.10.0...v0.11.0