Skip to content

v0.2.7 — the per-statement gap, closed layer by layer

Choose a tag to compare

@punnerud punnerud released this 03 Aug 01:21
· 109 commits to main since this release

Verified on Linux x86-64, macOS/Apple Silicon and Windows x86-64 — tests, clippy and the crash harnesses green on all three. Prebuilt binaries attached (CLI for all three platforms + the sqlite3 drop-in shim; sha256sums below).

The 0.2.6 known issue, closed — and two more the fix's review caught

  • The :memory: savepoint engine bug is fixed (0.2.6's named known issue, and the swap route's N4 report: btree corruption after ~4000 autocommit-plus-savepoint cycles, with silent wrong answers after). Private in-memory backing adopts committed pages in place — a third page state the savepoint machinery's COW dichotomy never covered. Savepoints now track adopted pages explicitly; the consumer's exact 4000-cycle repro is a regression test, judged by the page-accounting verifier.
  • A pre-existing silent wrong answer, found and fixed: a large (extent-backed) value deleted after a SAVEPOINT and overwritten by a later insert in the same scope could survive ROLLBACK TO with the new payload under the old row — committed, no error, in every release since extents shipped. Freed extent runs are now parked while a savepoint is open. Found by the adversarial review run on this release's savepoint work; reproduced byte-for-byte on 0.2.6 before fixing.
  • The same review caught the new savepoint journal's own blind spot (freed-then-recycled tree pages) before it ever merged — that repro is pinned too.

Performance: the per-statement gap, attacked layer by layer

The swap route's profiling put mpedb ~17–19 % behind stdlib on their suite under equal load, attributed to per-statement execute latency (345 µs vs 135 µs through Django). This release closes the structural share of that gap; every change was measured in isolation, A/B'd against its parent commit:

  • The in-transaction fast path (every ORM statement): the text-plan memo is now checked FIRST — a hit skips three per-statement text re-inspections — and a session revalidates a plan's table facts once instead of opening a fresh read snapshot per statement. Session-path overhead measured at ~2.1 µs/statement went to ~0; an in-transaction point-SELECT is no longer slower than its autocommit twin.
  • Transaction control never compiles: SAVEPOINT/RELEASE/ROLLBACK TO dispatch through the real grammar without the compile pipeline. Django mints a unique savepoint name per use — each one used to pay a full compile, churn the 512-entry text memo against the hot statements, and grow the plan cache without bound. Savepoint statements: 2× faster, and unique names now cost the same as repeated ones.
  • Savepoints capture lazily: SAVEPOINT used to copy every dirty page eagerly — Django's fixture pattern (fixture in the outer transaction, savepoint per test) paid O(fixture) per test. Pre-images are now journaled at first touch; the big-fixture savepoint cycle went from 3× the small-fixture cost to parity.
  • The Python DB-API layer dispatches in one pass (no per-statement lowercase scans or redundant copies), and executemany runs its rows on the Rust side of the GIL in 1024-row chunks.
  • LIKE scans: patterns with a wildcard-free core (%x%, x%, %x, exact) now run as allocation-free byte searches, and the general matcher no longer allocates per row — the LIKE '%x%' full-scan cell went from 5.4× to 3.0× stdlib. A 20 000-case differential test holds every fast path to the old matcher, verbatim.

Where mpedb already led — point/range reads on disk, empty transactions — nothing moved (those cells are the null controls).

Attached binaries

file contents
mpedb-0.2.7-linux-x86_64.tar.gz mpedb CLI, Linux x86-64
mpedb-0.2.7-macos-arm64.tar.gz mpedb CLI, macOS Apple Silicon
mpedb-0.2.7-windows-x86_64.zip mpedb.exe, Windows x86-64
libmpedb_sqlite3-0.2.7-linux-x86_64.tar.gz sqlite3 drop-in shim, Linux
libmpedb_sqlite3-0.2.7-macos-arm64.tar.gz sqlite3 drop-in shim, macOS
SHA256SUMS checksums for everything above