Skip to content

v0.2.2 — silent bulk-load data-loss fix (unified_memtable default OFF)

Choose a tag to compare

View all tags
@EvgeniyPatlan EvgeniyPatlan released this 19 May 22:20
· 59 commits to main since this release
v0.2.2
2f2d360

TidesDB MySQL plugin — v0.2.2

Correctness release on top of v0.2.1. One critical fix that changes a
default to make the engine safe out of the box, one latent silent-loss
fix in the bulk-commit path, and the reverse-ref/OSTAT handler fix that
landed on main between tags.

Tests: 61/61 MTR pass (the tidesdb_unified_memtable test now opts
into the unified path explicitly via a -master.opt so it stays
covered after the default flip). HammerDB 5.0 TPROC-C WARE=20 / BUILDVU=8
/ RUNVU=16 verified PASS out of the box (2511 NOPM, 0 handler-unsupported
errors, schema build + reverse-ref/OSTAT clean).

What changed

Correctness (CRITICAL — silent data loss)

  • Default tidesdb_unified_memtable=OFF. The TidesDB v9.2.0 unified
    WAL+memtable path silently loses committed rows under many concurrent
    writers. HammerDB TPROC-C at WARE=20/BUILDVU=8 dropped ~95% of
    bulk-loaded rows (MIN(o_id)=3001 across every district while
    district.d_next_o_id ~ 3000 — 100% of build-phase rows gone, only
    post-build single-row inserts survived). Every tidesdb_txn_commit
    returned TDB_SUCCESS and all loaders reported FINISHED SUCCESS.
    Refuted handler-side hypotheses (skip_list deep-copies key+value,
    ruling out txn_reset-after-commit corruption) and visibility lag
    (stable across 20s of repeated reads). Flipping the same handler
    binary to per-CF memtables produced a 100%-correct build. Until the
    engine fixes the rotation race, default the sysvar to OFF so
    correctness holds out of the box. Opt back in explicitly with
    tidesdb_unified_memtable=ON for low-concurrency multi-table OLTP.

    MTR coverage of the unified path is preserved via a per-test
    tidesdb_unified_memtable-master.opt that forces ON for that one
    test. Four result files were regenerated; their previous baselines
    encoded unified-mode degenerate stats (literal FAIL: DATA_LENGTH is 0 in tidesdb_info_schema, bogus rows=2 after ANALYZE that
    prevented MRR) — per-CF correctly reports non-zero data length,
    accurate row counts, and the optimizer now picks range/MRR scans on
    IN-lists.

Correctness (HIGH — latent silent loss in bulk path)

  • maybe_bulk_commit no longer silently swallows a failed mid-batch
    commit.     A failed tidesdb_txn_commit in the bulk insert/update/
    delete path was downgraded to an info log; tidesdb_txn_reset then
    discarded the batch's buffered ops, yet the function returned 0, so
    the caller saw success and the loader reported FINISHED SUCCESS
    while up to TIDESDB_BULK_INSERT_BATCH_OPS (500) rows per batch were
    silently lost. The fix retries transient resource errors (TDB_ERR_ CONFLICT / LOCKED / MEMORY_LIMIT) up to 4 attempts with a short
    backoff — tidesdb_txn_commit returns these errors before marking
    the txn aborted, so the buffered ops survive and re-committing is
    safe — then on non-transient errors or exhausted retries returns the
    mapped error without resetting the txn, so the SQL layer rolls the
    statement back instead of corrupting the table.

Correctness (HIGH — handler API)

  • Reverse-ref / OSTAT (ER_ILLEGAL_HA on ORDER BY pk DESC LIMIT 1).
    ha_tidesdb now implements index_read_last_map and fixes the
    partial-prefix HA_READ_PREFIX_LAST seek so that
    WHERE w_id=? AND d_id=? ORDER BY o_id DESC LIMIT 1 (the exact
    TPC-C Order-Status query) no longer surfaces error 1031. Verified
    via 6 dedicated MTR cases plus a full HammerDB 5.0 TPROC-C build+run.

Pull

docker pull evgeniypatlan/test-images:mysql-9.7-tidesdb-v0.2.2

Digest: sha256:7dc1d358470ed9c13e5cf31b189e61dc2ae0572a6b8557467454d172a0131bb3

Also tagged mysql-9.7-tidesdb-latest.

Assets 2
Loading