libdb 2026.09.10

A single-defect release, and the defect is the one that has been distorting libdb's
concurrency story: every transaction was acquiring and releasing 128 mutexes.

LOCK_LOCKERS took all 64 locker stripes, twice per transaction

src/dbinc/lock.h defined LOCK_LOCKERS as a serial loop over all 64
mtx_locker_stripe[] entries. __lock_getlocker takes it on every txn_begin and
__lock_freelocker on every txn_end — so each transaction acquired and released
128 mutexes, completely defeating the striping the structure was given. They block
in __db_pthread_mutex_condwait, a sleeping mutex, which is why the kernel's spinlock
slowpath dominated the profile.

Stripes are now keyed on the bucket index
(LOCK_LOCKER_STRIPE(LOCK_LOCKER_NDX(region, id))), so a stripe genuinely owns whole
locker_tab chains, plus mtx_locker_stripe[0] reused as the allocation latch for the
shared free-list and counters. 128 mutex operations per transaction → 4.

threads	before	after	delta	noise floor
1	95,324	123,672	+29.7%	±0.12%
8	94,581	116,201	+22.9%	±2.26%
32	84,178	115,288	+37.0%	±3.54%
96	67,290	102,377	+52.1%	±0.79%

Locker path 18.90% → 4.29% of profile. Kernel spinlock slowpath 3.22% → 0.01%.
And t=96 run-to-run CV 11.13% → 1.29% — the result is not merely faster but far
more predictable, which matters more than the mean for anyone sizing a system.

Independently reproduced on the pristine release build: 103,760 ops/s at t=96,
against 46,196 measured on the previous release before the fix.

Reusing stripe 0 is why your environments still attach

The allocation latch reuses an existing stripe rather than adding a region field.
src/env/env_sig.c hashes that struct, so a new field would change
__env_struct_sig() and every existing environment would refuse to attach with
BDB1539 — while abidiff stayed green, because the public ABI would not move. The
signature is unchanged at 0xb86f77f0.

Two diagnoses corrected on the way here

This release is worth reading for what it says about the previous diagnosis.

The recorded cause was wrong. P1 was documented as the PGNO_BASE_MD allocation
convoy — __db_new holding the metadata page write-locked to commit across its own
fsync. That hold is real and still true, but profiling showed __db_new does not
appear in the profile at all. The earlier "400 of 400 waits are on page 0" evidence
measured lock-manager waits, which are real but brief: st_lock_wait rises 0 →
14,986 from t=1 to t=96 while sampling during load finds zero standing WAIT
records. A wait that never forms a queue is not a convoy. Two prior attempts at P1
produced nothing, and aiming at the wrong subsystem is likely part of why.

The first proposed fix would have corrupted the locker table. Keying the stripe on
the locker id — the obvious reading of "just take one stripe" — is unsound: the bucket
is hash(locker) % locker_t_size where locker_t_size is a prime, while the
stripe was id & 63. Those mappings are independent, so two ids can share a bucket
chain and land in different stripes, and an id-keyed stripe does not own the chain its
holder walks. That is exactly why the original code took all 64. Keying on the
bucket is what makes single-stripe ownership true rather than merely plausible.

Tuning was ruled out, not assumed. set_mp_mtxcount=4096 and
set_lk_partitions=960 are both within noise at t=96, confirming a code defect
rather than a defaults problem. No knob relieves a hardcoded 64-mutex convoy.

What was deliberately not converted

__lock_getlocker, __lock_freelocker and __lock_id are converted.
addfamilylocker/familyremove (touch two buckets), id_free/id_free_pp (cold
paths), and sireap/deadlock/failchk/stat/sicleanup (genuine full-table walks)
still take all stripes, each with its reason recorded in
test/bench/P1-FIX-2026-09.md. A partial conversion that is provably safe beats a
complete one that is not.

The limit, stated plainly

Throughput still does not climb monotonically. The t=32 dip is removed (0.883× →
0.932× of peak) and t=96 now beats the previous build's best point at any thread
count — but the scaling-shape gate's concern is reduced, not retired, and
test/KNOWN-ISSUES.md marks P1 "fixed, partially" rather than closed. The baseline on
this hardware also did not reproduce the previously reported t=8 peak; it peaked at
t=1, and the report says so instead of quietly adopting the more flattering framing.

Known issues

Unchanged and open: P3 (DB_LOG_DIRECT — __log_write passes both an unaligned
buffer and an arbitrary length, so it needs its own durability argument), T1–T6,
S1 (opt-in DB_MPOOL_AIO), S5, U1–U6, W1, B1, and F1 (a
non-reproducible Apple clang crash, recorded with instructions to capture artifacts
before filing). See test/KNOWN-ISSUES.md.

Compatibility

No on-disk, log, region, or public-ABI change. __env_struct_sig() is 0xb86f77f0;
public sizes remain 1744/552/2088/336; the compatibility triplet stays 2026.0.9
and db_version() returns 2026,0,9, checked against a built library. Existing
environments attach unchanged.

Qualified from a pristine clone on a dedicated 96-vCPU EC2 instance: build, version and
ABI, environment signature, exec-bit gate, manifest self-check, db tier, leak tier, and
a fresh throughput measurement confirming the improvement on the release artifact
itself. ssi001–ssi011 11/11 and the DIAGNOSTIC lock-order checker did not fire.