Skip to content

Performance evidence

Kronik Maintainers edited this page Nov 11, 2026 · 2 revisions

Performance evidence

This page summarizes the performance measurements recorded for Kronik. The full methodology, the scripts that produce every number and the current targets are in docs/perf.md in the repository. Raw JSON receipts that accompanied early development were development artifacts and are not part of the public repository; the numbers below are the recorded results, and historical dates are given where they were recorded.

Test hardware

Development machine Test VPS
CPU AMD EPYC 9354P, 8 vCPU Intel Xeon Skylake, 2 vCPU
Memory 31 GB 4 GB
OS Debian 13 Debian 13, no /dev/kvm
4 KiB O_DSYNC write 1.08 ms 0.85 ms

The test VPS is the machine used for the release-verification runs of the later milestones; it is not a production host. The development machine was under load during some early runs, so those wall times are pessimistic; ratios between rows remain useful.

Committed turns on one node (M1, 2026-10-25)

Method: the release binary on a fresh data directory on a real disk, the counter test guest, and one load-generator connection per worker. Each request counts only after its turn record is fsynced and applied. Rows are 2 s of warm-up followed by 10 s measured; the 200 req/s row is open loop.

Workload Development turns/s Development p50 / p99 VPS turns/s VPS p50 / p99
1 actor, 1 client 181 4.9 / 12.8 ms 174 5.2 / 12.7 ms
1 actor, 16 clients 206 76.9 / 97.2 ms 190 82.7 / 114.3 ms
64 actors, 64 clients 7 842 7.6 / 17.6 ms 2 769 21.9 / 49.6 ms
256 actors, 256 clients 7 995 29.8 / 72.0 ms 2 774 91.6 / 167.9 ms
256 actors, open loop 200 req/s 200 5.1 / 11.8 ms 200 5.7 / 22.9 ms

A sequential acceptance run (10 000 increments per actor from 16 clients to one actor) took 45.9 s on the development machine and 58.4 s on the VPS.

Against the build brief's targets:

  • 1 000 committed turns/s for the Rust counter with group commit: the target is across actors, not per actor, and it is met (ADR 0111) — about 7 900 turns/s on the development machine and 2 770 turns/s on the 2-vCPU VPS. The per-actor rate is not a gate. One actor commits about 180–210 turns/s; each turn waits for the group-commit window, one fsync and the apply before the actor's next turn starts.
  • Gateway p99 below 25 ms at 200 req/s: met — 11.8 ms on the development machine and 22.9 ms on the VPS, where the margin is small because the load generator and the daemon share the two vCPUs.

Replay speed (M3)

Method: record a reference workload (12 000 POST /counter/.../increment requests at 200 req/s over 256 actors for a 60 s production window), then replay it with the same code and time an isolated replay on several workers, a continuous replay, and both at once. The target is replay at 20x real time or more on the test VPS.

Run (test VPS) Isolated Continuous Both
2026-10-25, first measurement 35.3x 28.8x 19.1x
2026-10-25, second measurement 35.1x 30.0x 19.4x
2026-10-26, improved build, run 1 40.9x 27.2x 23.6x
2026-10-26, improved build, run 2 40.4x 28.6x 22.4x
2026-10-26, improved build, run 3 34.8x 27.1x 24.3x

The first build met 20x in isolated and continuous replay but not when both ran together. After a profiled change to the Wasm pooling allocator, applying turns and replay scheduling, all three modes met 20x in three consecutive runs. On the development machine, isolated replay measured 25x to 47x, continuous 23x to 42x and both 20x to 31x. Isolated replay scales to about 1.6x on two workers and 1.85x on four, and not beyond; that limit was not investigated further.

Startup after a crash (M6)

The build brief asks for a start with a 1 GiB log after the latest snapshot to take under 30 seconds. Measured with 1 GiB of log after the snapshot (3 002 369 records):

Build Without state.db state.db at the snapshot
First catch-up (2026-10-25) 30.8–32.8 s —
Streamed catch-up, 2026-10-26 18.8–19.7 s 16.9–17.2 s

The first attempt did not meet the target. The fix was a streamed catch-up that decodes segments on separate cores while the applier writes prepared rows, plus bulk index building; the target is now met at about 19 s. The replay does not change SQLite's page size, and a crash during replay cannot leave a torn state.db behind: a marker file causes the database to be rebuilt from the snapshot and the log on the next start.

Privacy-mode overhead (M8)

The build brief asks that the off mode have no measurable overhead compared with the previous release. Three workloads (counter, send and notes) were measured alternately on both builds, five runs each, niceness -10, on the development machine:

  • off against the previous release: no measurable overhead. Every difference was within the run-to-run noise: throughput +2 % to +20 %, p99 -18 % to -2 %, CPU per record -24 % to +13 %, against standard deviations of 8 % to 36 %. The log was byte for byte the same size.
  • on costs log size more than time. Each sealed value carries a key version, a 24-byte nonce and a 16-byte tag: +32 % log bytes per record for counter, +34 % for send and +57 % for notes. Throughput stayed within the noise; CPU per record rose by up to 42 % for notes.

Crash test, deterministic simulation and nightly volume

The v0.1 definition of done asks for 1 000 kill -9 cycles and for DST over 10 000 seeds plus a run of at least 1 000 000 steps, all without invariant violations. The recorded runs (2026-10-25) met these:

Check Result Wall time
Crash test, 1 000 cycles No violations; 118 915 acknowledged requests, 2 451 unsafe effects and none sent twice, 477 keyed effects redone after a restart, final identity replay of 124 782 inputs empty 49 min
DST, seeds 0–9 999, counter, effects and webhook workloads, each twice 30 000 runs: no violation, no nondeterminism 735 s, 8 processes
DST, seeds 0–9 999, webhook with the TypeScript relay, each twice 10 000 runs: no violation, no nondeterminism 1 253 s, 8 processes
DST long run, seed 0, twice 1 003 063 steps: no violation, the same trace both times 362 s per run

Public single-node benchmark runner (2026-10-31)

scripts/benchmark-single-node.py drives an already-built real daemon through its public CLI, API and gateway, and records a versioned kronik.benchmark/v1 report with binary, guest, generator and rig identities. scripts/check-performance.py compares two such reports and fails a change greater than 10 % in either direction; exactly 10 % is allowed. The comparator's synthetic enforcement tests pass, but they are not performance measurements.

Recorded single-node results:

Run Committed turns/s Gateway p99 at 20 req/s Identity replay (both modes) Snapshot
Debug functional smoke 726.3 8.91 ms 5.38x 32.5 ms
Released v0.3.0 build control 1 002.2 4.72 ms 18.45x 16.3 ms

These are short same-host controls, not physical M32 acceptance, and the two runs are not comparable as a regression baseline because the binary and generator provenance differ.

The one historical between-release comparison (v0.1.0 against v0.3.0, same host, guest, generator and workload) exercised the comparator with real reports:

Measured control v0.1.0 v0.3.0 Gate
Committed counter turns/s 996.503 926.867 -6.99 %, pass
Gateway p99 5.111807 ms 6.160383 ms +20.51 %, regression
Single-node replay ratio 19.8029x 19.6047x -1.00 %, pass
Snapshot creation 16.411 ms 16.480 ms +0.42 %, pass

The comparator correctly refused the more-than-10 % gateway regression. The comparison is recorded as a failed gate, was not rerun until green, and neither the threshold nor the workload was relaxed. It is not a causal diagnosis, a latest-product baseline or physical M32 acceptance.

What is not covered

  • The M32 cluster targets are not measured. They require three separate nodes, each 4 vCPU and 8 GB: at least 20 000 committed turns/s, 5 000 requests/s with p99 below 50 ms, distributed replay at least 50 times the recorded production window, leader replacement within 10 s of node loss, and 100 GB of state per node including snapshots and compaction. Three processes on one host are not three machines or failure zones, so no result here closes those targets, and no target is adjusted without an ADR and actual measurements.
  • The cluster benchmark depends on the cluster track (M16–M19), which is not implemented yet; the single-node runner does not synthesize a cluster result.
  • The development-machine runs were taken under variable load; they are useful for ratios and ordering, not as absolute hardware performance.

The mapping from the build brief's targets to the repository, including the remaining gaps, is in the compliance audit.

Clone this wiki locally