v0.4.1 -- perf-instrumentation infrastructure

v0.4.1 — perf-instrumentation infrastructure release

Layer-by-layer plugin perf capture surface. No engine optimisations ship in this release — the deliverable is the measurement surface that the v0.5.0 cycle will optimise against. The bundled engine is unchanged from v0.4.0 (still TidesDB v9.3.2, still shipped unpatched).

The default perconalab/tidesdb-mysql:0.4.1 image is a byte-identical re-stamp of the v0.4.0 release image: with the default TIDESDB_PERF=0 build, every TDB_PERF_SCOPE(...) macro expands to ((void)0), so the default user-facing code path is unchanged. Instrumentation lives in a separate, opt-in perconalab/tidesdb-mysql:0.4.1-perf variant.

What's new

• TDB_PERF_SCOPE(MethodId) RAII macro across the full plugin entry surface (32 MethodId values: write_row, update_row, delete_row, index_read_map, index_next, index_prev, rnd_next, rnd_pos, external_lock, start_stmt, store_lock, commit, rollback, four savepoint_*, open, close, info, table_flags_cache_init, create, delete_table, four inplace-ALTER virtuals, serialize_row, deserialize_row, key_copy_to_comparable, pk_from_record, encrypt_row_into, decrypt_row). On capture-off the overhead is two __rdtsc() reads + one atomic load per scope. Compile-time-gated on TIDESDB_PERF.
• Thread-local rdtsc ring buffer (default 2^16 samples per thread, runtime-configurable). 24-byte Sample packed natural-alignment with static_assert on the layout. Rings self-register into a lock-free linked list (g_rings_head) on first use.
• Background flusher thread pinned to CPU 0; walks the ring list every tidesdb_perf_flush_interval_ms, bucket-sorts samples by method, pwritevs each bucket to its method's append-only .bin file. Calibrates TSC via 20 ms std::chrono::steady_clock spin and writes the calibration to meta.json.
• Four perf sysvars (all gated on TIDESDB_PERF=1): tidesdb_perf_capture (BOOL, default OFF — the kill-switch), tidesdb_perf_output_dir, tidesdb_perf_ring_capacity_pow2, tidesdb_perf_flush_interval_ms.
• tools/tidesdb_perf_analyze — Python 3 + numpy offline analyser. Parses .bin files (struct.pack('<BBH4xQQ', ...)), aggregates per-method calls / total_ms / mean_us / p50 / p95 / p99 / max, emits markdown. --compare A B produces a side-by-side diff.
• bench/perf/run-perf-capture.sh integration harness. Wraps bench/hammerdb/run-hammerdb.sh, forces perf sysvars ON, mounts the perf output directory into the container, copies + chowns artifacts back to host, runs the analyser.
• docker/patches/tidesql/0001-perf-instrumentation.patch (880 lines) — ports Sample / TLS_Ring / PerfScope + 30 TDB_PERF_SCOPE call sites into TideSQL's ha_tidesdb.cc for the eventual MySQL-vs-MariaDB side-by-side run.
• 5 MTR perf tests + 8 gtest ring tests + 2 analyser pytests — all green.

Headline numbers (WARE=10 RUNVU=8 1m+3m HammerDB TPROC-C, perf ON)

Captured from the perf variant on the validation host:

method	calls	mean_us	p99_us	max_us
write_row	5,294,113	16.54	17.13	72,783.99
index_read_map	1,900,638	13.19	108.09	18,306.95
deserialize_row	3,950,283	0.90	3.61	1,034.43
serialize_row	6,011,734	0.41	2.32	4,298.43
pk_from_record	6,361,758	0.19	0.72	7,047.85

Throughput with perf ON: 1841 NOPM (vs ~2028 NOPM baseline → ~9% capture overhead).

write_row p99 = 17 μs but max = 72 ms — long compaction-stall tail dominates total. index_read_map p95 (44.7 μs) is 8× p50 (6.1 μs) — read path also has a meaningful long tail. These are the v0.5.0 optimisation entry points.

Validation

Full validation matrix green — see docs/v0.4.1-validation-report.md.

Docker

Default image (no perf, equivalent to v0.4.0):
```
docker pull perconalab/tidesdb-mysql:0.4.1
```

Perf variant (opt-in; instrumented build for the v0.5.0 optimisation cycle):
```
docker pull perconalab/tidesdb-mysql:0.4.1-perf
```

Deferred to v0.5.0

• Engine-side optimisations driven by the captured hotspots above.
• sut-mariadb-tidesdb:9.3.0-perf SUT image build (TideSQL patch is committed; image not yet built).
• TideSQL perf sysvars (the patch wires scopes + ring; sysvar wiring deferred).