Conversation
Forward-port groundwork for Cahill's SSI (cf. v4.6.21-SSI) onto 5.3.x. Additive interface surface only; no behavior change yet: - db.in: DB_LOCK_SIREAD=9 lock mode; internal TXN_SNAPSHOT_SAFE=0x80000; public errors DB_SNAPSHOT_CONFLICT (-30968), DB_SNAPSHOT_UNSAFE (-30967). - txn.h: TXN_DTL_WCONF/RCONF rw-antidependency markers on __txn_detail; mtx_oldlsn/old_lsn oldest-active-LSN fields on the txn region. - db_err.c: db_strerror() messages for the two new error codes. Error numbers/flag bits chosen for the 5.3.x layout (the prototype's -30971/-30970 are already used on master). Builds clean (configure + make). Subsequent milestones (gated): SIREAD lock mode + conflict matrix (M2), read/write tracking hooks (M3), dangerous-structure detection + abort (M4), and the TCL test campaign (M5).
- lock_region.c: grow conflict matrix to 10 modes; SIREAD (mode 9) is a non-blocking 'soft' lock (all-zero row/column) used only to record rw-antidependencies, never to block. - lock.h: per-object `sireaders` list; `td_off` link from locker to its TXN_DETAIL; DB_LOCKER_FREED flag (keep freed lockers alive while their SIREAD locks persist); LOCKER_TD/LOCK_HOLDER/LOCK_OWNER/LOCK_READLSN/ LOCK_COMMITLSN helpers over MVCC read_lsn/visible_lsn. Builds clean. Next (M2 part 2): SIREAD acquisition + rw-conflict recording in __lock_get_internal, __lock_sicommit/__lock_sicleanup GC against __txn_oldest_reader — adapted to 5.3.x partitioned lock regions.
Documents how Cahill's single-global-table SIREAD GC maps onto 5.3.x partitioned regions. Key decision: do not free lockers while holding a partition mutex (avoids partition->mtx_lockers lock-order inversion); collect victims and free after releasing the partition mutex. Reuse the existing __txn_oldest_reader; compute it before taking any partition mutex.
- lock_id.c: init locker->td_off = INVALID_ROFF (txn layer links it to the TXN_DETAIL when a transactional locker is created). - lock.c: SH_TAILQ_INIT the per-object sireaders list on object creation. Pure initialization; no behavior change. Builds clean. Next (part 2b): safe_si-gated SIREAD acquisition + rw-antidependency scan in __lock_get_internal, then __lock_sicommit/__lock_sicleanup GC per the M2 design note.
- lock.h: internal DB_LOCK_SNAPSHOT_SAFE acquire flag (0x020000, free bit, clear of the interface-flag range). - lock.c: parse safe_si from DB_LOCK_SNAPSHOT_SAFE and clear it at entry; add the same-locker early-out (holder already owns WRITE and requests SIREAD -> grant trivially). Behind the opt-in flag; no change to the default path. Builds clean. Next (2b-ii): place SIREAD new-locks on obj->sireaders; then __lock_sicommit/__lock_sicleanup GC.
Granted SIREAD locks (safe_si) are linked on obj->sireaders instead of obj->holders, while remaining owned by the locker (heldby) so they are iterated/released at txn end. SIREAD never conflicts, so it always reaches GRANT. Builds clean. Next: __lock_sicommit (detach at commit) and __lock_sicleanup GC against __txn_oldest_reader.
Add __lock_sicleanup (PUBLIC) + __lock_siclean_obj: sweep objects and free SIREAD markers whose owning txns committed and whose snapshots fall outside the oldest active reader's window. Per the design note: __txn_oldest_reader is computed once before any partition mutex; freeing happens under the per-partition mutex via __lock_freelock(UNLINK|FREE); no locker is freed under a partition mutex. lock.c now includes dbinc/txn.h for TXN_DETAIL. Not yet invoked (wired at M4); builds clean.
- dist/api_flags + regenerated api_flags.in: public DB_TXN_SNAPSHOT_SAFE (0x00000080) via dist/s_apiflags (not hand-edited). - txn.c: accept DB_TXN_SNAPSHOT_SAFE in txn_begin; it implies snapshot isolation (TXN_SNAPSHOT) and sets internal TXN_SNAPSHOT_SAFE; children inherit it. Link each txn's locker to its TXN_DETAIL via td_off so the lock layer can read the owner's MVCC snapshot LSNs. Builds clean (configure regenerates db.h with the new public flag).
In __lock_get_internal, a snapshot-safe WRITE that finds SIREAD markers from concurrent/overlapping snapshot readers records the rw-edge: set TXN_DTL_RCONF on the reader (out-edge) and TXN_DTL_WCONF on the writer (in-edge). A txn that becomes both ends (a pivot) is aborted with DB_SNAPSHOT_UNSAFE. A writer upgrading its own SIREAD drops that marker. Obsolete-marker reclaim is left to __lock_sicleanup (avoids taking the txn system lock under a partition mutex, per the design note). Builds clean.
si_ref on TXN_DETAIL parallels mvcc_ref; detail/locker outlive the reader's commit until the last SIREAD marker is GC'd. Documents the cross-subsystem last-reference free that the M5 campaign must stress.
Add si_ref to TXN_DETAIL (parallels mvcc_ref) and bump the owner's si_ref when a SIREAD marker is granted, so the detail can be pinned past commit per the M4 lifecycle note. Builds clean.
- ssi001.tcl: canonical SSI write-skew test; x and y in separate DBs so only the read/write antidependency contends (not page locks); short lock timeout so the single-threaded interleave fails fast instead of hanging. - tcl_txn.c: expose -snapshot_safe to the TCL txn command (DB_TXN_SNAPSHOT_SAFE). - testparams.tcl/TESTS: register the ssi suite. Baseline (pre-implementation) outcome: FAILS as designed -- ssi_no_write_skew expected 1 got 0 (both txns commit). This is the gate that M2-finish + the mp_fget read hook must turn green.
… abort Turns the ssi001 write-skew gate GREEN. End to end: - db_meta.c (__db_lget): snapshot-safe reads acquire a SIREAD marker (READ->SIREAD + DB_LOCK_SNAPSHOT_SAFE); snapshot-safe writes are flagged so the lock manager runs the rw-antidependency scan. Plain snapshot behavior is unchanged. - lock.c: __lock_sicommit detaches SIREAD markers at commit (persist on sireaders, locker flagged DB_LOCKER_FREED) and releases them on abort. - lock_id.c: defer freeing a DB_LOCKER_FREED locker while markers remain. - txn.c: init td->si_ref; link locker->td_off; retain TXN_DETAIL while si_ref>0; call __lock_sicommit at txn end; commit-time pivot check aborts a transaction holding both TXN_DTL_WCONF and TXN_DTL_RCONF with DB_SNAPSHOT_CONFLICT. - txn.h: drop the unused region oldest-LSN fields (__txn_oldest_reader, which already exists on 5.3.x, uses a local). Verified: ssi001 PASSES; plain -snapshot still allows write-skew (both commit); -snapshot_safe aborts exactly the pivot. NOTE: periodic __lock_sicleanup GC of persisted markers is not yet wired (markers are reclaimed at env close); to be added before heavy/long-running M5 runs.
…time - txn_chkpt.c: call __lock_sicleanup at each checkpoint to reclaim SIREAD markers of committed readers past the oldest-reader horizon. - lock.c: __lock_siclean_obj frees committed-reader markers without DB_LOCK_UNLINK (they were detached from heldby at commit) and accounts for si_ref/nlocks by hand. - txn_region.c: the MVCC detail-free now also requires td->si_ref == 0 so a detail with outstanding SIREAD markers is never freed out from under them. Verified: 200-iteration write-skew stress with periodic checkpoints -> 200 commits / 200 aborts, clean close, no crash; ssi001 still passes. KNOWN LIMITATION: a committed reader's locker+detail structs persist until env close (markers/lock structs are reclaimed; the per-reader locker/detail reclaim pass is a follow-up). Safe (no UAF), bounded; tracked in ROADMAP.
Verifies SSI commits non-conflicting work: disjoint read/write sets, a read-only snapshot-safe txn, and read-then-write on one item by a single txn all commit. Guards against over-aborting. ssi001 + ssi002 both pass.
gburd
marked this pull request as ready for review
June 16, 2026 12:40
gburd
added a commit
that referenced
this pull request
Jul 31, 2026
feat(ssi): Serializable Snapshot Isolation for 5.3.x
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Forward-port Michael Cahill's Serializable Snapshot Isolation (SSI) onto the 5.3.x
master. The faithful 4.6.21 prototype is preserved as thev4.6.21-SSItag/release; this PR re-implements that algorithm against 5.3.x (the lock manager, mpool, and txn subsystems diverged substantially, so it is a reimplementation guided by the prototype, not a patch port)."Production ready" bar: correct, opt-in
DB_TXN_SNAPSHOT_SAFEmode layered on existing multiversion (SI), validated by the TCL test suite plus new SSI anomaly tests.Staged, test-gated milestones
DB_LOCK_SIREADmode, internalTXN_SNAPSHOT_SAFE, errorsDB_SNAPSHOT_CONFLICT/DB_SNAPSHOT_UNSAFE,TXN_DTL_WCONF/RCONFmarkers, txn-region oldest-LSN fields,db_strerrormessages. Builds clean (configure + make).DB_SNAPSHOT_CONFLICT) and publicDB_TXN_SNAPSHOT_SAFEwiring intotxn_begin.Draft until M5 is green. Each milestone is build/test-gated and reviewed before the next.
Provenance
Algorithm: Cahill, Röhm & Fekete, Serializable Isolation for Snapshot Databases, SIGMOD 2008. Prototype:
v4.6.21-SSI(upstream commit 1405db1). Never shipped in Berkeley DB; later adopted by PostgreSQL 9.1.