Skip to content

v0.2.3 — WAL durability fix (silent loss on SIGKILL)

Choose a tag to compare

@EvgeniyPatlan EvgeniyPatlan released this 20 May 18:09
· 58 commits to main since this release

TidesDB MySQL plugin — v0.2.3

Critical durability fix on top of v0.2.2. The vendored TidesDB v9.2.0 engine silently lost every committed write on a hard process crash (SIGKILL / power-loss / kernel panic) even with the plugin requesting sync_mode=FULL — pre-kill COUNT showed the data, post-restart COUNT was 0. MySQL's own data dictionary survived; the engine's per-CF WAL was being restored to header-only across the kill+restart cycle.

Tests: 61/61 MTR pass (no regressions from the patch). 5/5 INSERTs survive docker kill -9 + restart; mixed 100-row BEGIN/COMMIT + 5-row autocommit → 105/105 recovered post-SIGKILL.

What changed

Correctness (CRITICAL — silent data loss on crash)

Bundles three stacked engine patches against the vendored TidesDB v9.2.0 as a single unified diff at docker/patches/0001-walfix.patch, applied to the upstream clone in docker/Dockerfile.mysql before the engine cmake. The patch will be removed once these fixes land upstream in TidesDB.

  1. src/block_manager.c convert_sync_mode() inverted case logic. The function maps the engine's tidesdb_sync_mode_t enum (NONE=0, INTERVAL=1, FULL=2) to the block manager's two-mode enum (NONE=0, FULL=1). Upstream: case 1 returned BLOCK_MANAGER_SYNC_FULL (mapping INTERVAL → FULL) and case 2 fell through defaultBLOCK_MANAGER_SYNC_NONE (mapping FULL → NONE). So a sync request of FULL silently got NONE: the block manager opened the WAL without O_DSYNC and skipped the per-block fdatasync, leaving the write only in the kernel page cache.

  2. Multiple WAL block_manager_open sites in src/tidesdb.c passed the raw engine enum. The unified-memtable path (lines 17269, 23298), per-CF path (line 18600), and rotation reopens (lines 19174, 19190) passed config->sync_mode / cf->config.sync_mode / umt_sync_mode directly to block_manager_open without going through convert_sync_mode(). With (1) fixed, these still wired the wrong enum value into the block manager, so the fix had to extend to every WAL-open call site.

  3. tidesdb_create_column_family unconditionally truncated the WAL. The function is invoked both for fresh CREATE TABLE and during database open when an existing CF directory is rediscovered on disk. It called block_manager_truncate(new_wal), which wipes the WAL to header-only — running before recover_wals had a chance to replay it. So with sync now working, recovery still saw an empty WAL. Replaced with block_manager_validate_last_block(PERMISSIVE), which writes the header for a 0-byte file, leaves a valid header-only file alone (fresh CF case), and forward-scans + sets current_file_size to the last valid block for an existing WAL with data (recovery case). Same observable behaviour as truncate for fresh, preserves data for recovery.

Pull

docker pull evgeniypatlan/test-images:mysql-9.7-tidesdb-v0.2.3

Digest: sha256:d2a80d515d392f93975bde76ed3d9d19d5c64744521a5114c5bd180e57108341

Also tagged mysql-9.7-tidesdb-latest.