v0.2.4 — bug #4 patched: full row recovery on multi-SSTable post-restart
TidesDB MySQL plugin — v0.2.4
Patches the residual durability bug v0.2.3 left localized. All four engine bugs found during the investigation are now fixed.
Tests: 61/61 MTR pass. Post-SIGKILL row recovery, HammerDB WARE=10 BUILDVU=4 RUNVU=4 + 30s NewOrder mix:
| Table | v0.2.3 (broken) | v0.2.4 (fixed) | Expected |
|---|---|---|---|
tpcc__orders per (w,d) |
2,017 for d=1, 0 for d=2..10 | 3,145–3,186 each, all 10 districts | d_next_o_id − 1 |
tpcc__orders total |
~307k (full-scan only) | 309,943 | ~310k |
tpcc__order_line total |
2,669 | 3,100,951 (×1,162 recovered) | ~3.1M |
tpcc__stock total |
717 | 1,000,000 (×1,394 recovered) | 1M |
tpcc__customer total |
300,000 | 300,000 | 300k |
What changed
Correctness (CRITICAL — partial silent loss on SIGKILL with multi-SSTable level 1)
Bug #4: SSTable cursor cached the wrong block_size. Four sites in tidesdb/src/tidesdb.c (tidesdb_merge_source_advance and two paths in tidesdb_iter_seek_sstable_source_forward) set cursor->current_block_size = bdata_size / = block_data_size plus cursor->block_size_valid = 1 after consuming a cached SSTable block. The value cached was the cache-entry size (decompressed block data plus appended per-entry index entries), not the on-disk block size that block_manager_cursor_next needs to add to current_pos. On the next cursor_next call, current_pos jumped by header + bdata_size + footer — which we measured at 160 MB / 318 MB / 812 MB / 1.2 GB / 1.7 GB per call. After 2–3 calls, current_pos exceeded klog_data_end_offset and the cursor returned TDB_ERR_NOT_FOUND, dropping the rest of the SSTable.
The bug fired only when post-restart level 1 contained two SSTables — one loaded from disk via tidesdb_sstable_load (the pre-kill flushed SSTable) plus one created during recovery via tidesdb_level_add_sstable (the recovery-flushed memtable). Tables with one SSTable were unaffected — that's why customer and (in some test profiles) orders returned correct counts while order_line and stock returned 0.07–0.36% of their rows.
Fix: removed the four wrong current_block_size / block_size_valid = 1 assignments. cursor_next now pread's the real 4-byte on-disk size header per block transition. One syscall per block, in the host page cache anyway — measurable but negligible cost.
Patch grew from 99 → 155 lines
docker/patches/0001-walfix.patch now covers all four engine fixes (v0.2.3's three plus v0.2.4's one), applied to the upstream TidesDB v9.2.0 clone in docker/Dockerfile.mysql before cmake. Removable once equivalent fixes land upstream.
How to verify after a future TidesDB upgrade
cd bench/hammerdb
./recovery-diag.sh
# bench/results/recovery-diag-*/snapshots.txt — post-restart counts
# must match district.d_next_o_id - 1 for each (w, d).bench/hammerdb/run-all.sh runs the full v0.2.3/v0.2.4 suite (correctness baseline, recovery, VU sweep, head-to-head vs InnoDB, TPROC-H, sustained) and generates a self-contained REPORT.md.
Pull
docker pull evgeniypatlan/test-images:mysql-9.7-tidesdb-v0.2.4
Digest: sha256:b4e226acb6713f43652d676fbf63defd28e7cbd3ed56d9da9b64ea9fb3736740
Also tagged mysql-9.7-tidesdb-latest.