v0.9.5 — Perf Optimization

[jamesgober]

mod-alloc v0.9.5 — Tier 2 perf optimisation (~32x speedup)

Date: 2026-05-18
Compare: v0.9.4...v0.9.5

Headline

Tier 2 (backtraces) per-allocation overhead dropped from
~2,050 ns to ~57 ns on a Windows x86_64 dev host — roughly 32x
faster, comfortably under the REPS section 6 target of <200 ns
of Tier 2 overhead. The dominant cost was the per-thread arena
layer that buffered captured events between flushes; v0.9.5
removes the arena entirely and writes each captured event
straight to the global aggregation table.

Both tiers now sit well inside their REPS targets with
significant headroom for production use.

What changed

Bench numbers

Build	v0.9.4 (Windows x86_64)	v0.9.5 (Windows x86_64)	Speedup
Tier 1 only (counters, default)	45.5 ns	45.5 ns	—
Tier 1 + Tier 2 (backtraces)	~2,051 ns	~56.9 ns	~32x

Bench command:
cargo run --release --features backtraces --example bench_overhead

REPS section 6 targets: Tier 1 <50 ns total, Tier 2 <200 ns of
additional overhead. Both clear.

Removed: the per-thread arena layer

src/backtrace/arena.rs is deleted. Previously, each captured
event went through:

1. record_event(size) in src/backtrace/mod.rs
2. TLS lookup of the per-thread ARENA slot.
3. Lazy mmap/VirtualAlloc of a 64 KB arena page on first
   event (one-time per thread).
4. 72-byte memcpy of the captured Frames into the arena's
   entry struct at the cursor.
5. Cursor increment.
6. On every 512th event: synchronous flush of all 512
   entries into the global table via table::record.

Steps 2, 4, and 6 carried real cost on Windows where
thread_local! access is ~30–60 ns per lookup and an unbatched
memcpy of 72 bytes is ~10 ns. The flush was bounded
(~70–100 ns per entry × 512 entries = ~50 µs per flush,
amortising to ~100 ns per event), so the visible per-event cost
was dominated by TLS and memcpy.

The arena's rationale was contention reduction under concurrent
allocation. Measurement showed that table::record's
steady-state matching path is just two atomic operations
(fetch_add on count and total_bytes) on the bucket the
event hashes to. Linear probing in the 4,096-bucket default
table spreads writes across cache lines well; the per-thread
arena's batching no longer paid for itself once buckets were
warm (which happens after the very first event per call-site
per process).

Architecture after v0.9.5

ModAlloc::alloc(layout)
  ├─ System.alloc(layout)
  ├─ ReentryGuard::enter()   ← 1 TLS lookup (reentrancy flag)
  ├─ record_alloc(size)      ← 6 atomic ops (Tier 1 counters)
  ├─ register_self()         ← 1 atomic load (branch-predicted)
  └─ backtrace::record_event(size)
       ├─ current_fp()            ← 1 inline asm
       ├─ current_stack_bounds()  ← 1 TLS lookup (cached)
       ├─ walk(fp, bounds)        ← bounded 8-frame walk
       └─ table::record(...)      ← 2 atomic ops (matching path)

Tier 2 hot path: 2 TLS lookups total (down from 3), and no
inter-event buffering. The 72-byte arena memcpy and the
periodic flush blockage are both gone.

Inlining hints

• table::record is now #[inline] — without this, thin-LTO
  could not stitch the cross-module call into record_event.
• current_stack_bounds is now #[inline(always)] — the cache-
  hit fast path folds into the calling record_event body.
• ensure_init in table.rs is split into a hot
  #[inline(always)] accessor (three atomic loads for the
  steady state) and a cold #[cold] #[inline(never)] slow path
  that handles the one-time-per-process table allocation.

Together these let the optimizer reduce record_event to a
near-straight-line function inside ModAlloc::alloc.

Removed items

• src/backtrace/arena.rs (deleted)
• ENTRIES_PER_ARENA constant
• ArenaState, ArenaSlot structs
• The ARENA thread-local
• arena::record_event, arena::flush_current_thread
• The record_and_flush_round_trip arena unit test
• The super::arena::flush_current_thread() call inside
  table::call_sites_report (there is nothing to flush)

Test additions / changes

• Removed: the 1 arena unit test
• Existing tests unchanged. Full suite: 96 tests pass
  (52 unit + 1 + 1 + 1 + 1 + 1 + 7 + 4 + 1 + 1 + 5 + 1 + 1 + 19
  doctests) on cargo test --all-features.
• No behaviour changes to any test that exercises
  call_sites(), symbolicated_report(), dhat_json_string(),
  write_dhat_json(), or any dhat_compat::* API.

Design notes

Why the arena existed in the first place

v0.9.1's design (.dev/DESIGN_v0.9.1.md section 2) anticipated
that direct writes to a global table would contend badly under
multi-thread alloc storms — the same call site from N threads
would all CAS-claim the same bucket on the very first event,
then all hammer the bucket's count/total_bytes atomics. A
per-thread arena was the hedge: each thread accumulates locally,
then a single thread does the bulk insert.

The hedge turned out to be unnecessary. After the first event,
the bucket's count/total_bytes are uncontended fetch_adds
on a single cache line. Reading the bucket's hash field
(Acquire-ordered) is also uncontended after publication.
Modern x86_64 handles uncontended atomic adds at ~5–10 ns each;
under contention they slow to ~30–50 ns per op, but linear
probing in a 4,096-bucket table means contention only happens
when N call sites hash to the same bucket and N threads hit
them at the same time. Measured: no observable contention spike
in the 32-thread concurrent stress test
(tests/backtrace_concurrent.rs).

Why not also combine the remaining TLS slots

After the arena removal, two TLS slots remain on the hot path:
IN_ALLOC (reentrancy) in lib.rs and CACHED (stack bounds)
in backtrace/stack_bounds.rs. Combining them into a single
TLS struct would save one TLS lookup per allocation — at most
30–60 ns on Windows. The bench already shows we are 17 ns over
Tier 1 (which itself uses IN_ALLOC), so the actual remaining
Tier 2-specific TLS cost is at most ~30 ns. Combining would
require either touching the IN_ALLOC flag from inside the
backtrace module (layering inversion) or duplicating it. Not
worth the architectural cost for ~30 ns; revisit if a future
target tightens further.

Why #[inline(always)] and not #[inline] everywhere

#[inline] is a suggestion; #[inline(always)] is a directive.
For functions on the alloc hot path that are called from exactly
one place in release builds (current_fp, record_event,
current_stack_bounds's fast path), #[inline(always)] is
correct — there is no risk of code bloat from multiple
expansions because there are no multiple expansions. For
functions called from a few places (table::record), plain
#[inline] lets the compiler decide.

ensure_init_slow and query_os (the cfg-specific OS callers
in stack_bounds.rs) are #[cold] #[inline(never)]. They run
exactly once per process and once per thread respectively; we
want them out of the hot path's icache.

Migration

None. No public API changes. No new features. No new
dependencies. MSRV unchanged (1.75).

The internal hot-path restructure is invisible to callers:

• ModAlloc::call_sites() returns the same data
• ModAlloc::symbolicated_report() returns the same data
• ModAlloc::dhat_json_string() / write_dhat_json() produce
  the same JSON
• dhat_compat::HeapStats::get() / Profiler::* / Alloc
  surface unchanged
• MOD_ALLOC_BUCKETS env var still configures the table at the
  same level (4,096 buckets default)

Verification

Full matrix run on Windows host (x86_64):

• cargo build (default features): ✓
• cargo build --no-default-features: ✓
• cargo build --features counters: ✓
• cargo build --features backtraces: ✓
• cargo build --features symbolicate: ✓
• cargo build --features dhat-compat: ✓
• cargo build --all-features: ✓
• cargo +1.75 build --all-features (MSRV): ✓
• cargo fmt --all -- --check: ✓
• cargo clippy --all-targets -- -D warnings: ✓
• cargo clippy --all-targets --all-features -- -D warnings: ✓
• cargo clippy --all-targets --no-default-features -- -D warnings: ✓
• cargo doc --no-deps: ✓
• cargo doc --all-features --no-deps: ✓
• cargo test --all-features: 96/96 pass
• cargo run --release --features backtraces --example bench_overhead:
  56.9 ns / cycle (down from ~2,051 ns)

CI matrix: ubuntu-latest, macos-latest, windows-latest plus the
ASAN nightly Linux job. No new CI step required; existing
--features backtraces test step already exercises the hot
path. The 32-thread concurrent stress test
(tests/backtrace_concurrent.rs) passes unchanged, confirming
the direct-write path scales under multi-thread allocation.

Version-numbering note

mod-alloc has no v0.9.5 entry corresponding to the
"dev-bench swap" milestone in the roadmap — that milestone
shipped as dev-bench v0.9.7 (the consumer-side swap lives in
the dev-bench repo, not mod-alloc). This release reuses the
v0.9.5 slot for mod-alloc's own published version sequence on
crates.io: 0.9.4 → 0.9.5.

Follow-ups

• v1.0.0 — Stable API. Now the next milestone. The Tier 2
  perf target was the last REPS section 6 gating item before
  freeze. v1.0.0 prep: full public-API rustdoc audit, freeze
  the wire format, confirm zero-friction with dev-bench v0.9.7
  (the live consumer).
• Combine IN_ALLOC + CACHED TLS slots. Worth ~30 ns but
  costs architectural cleanliness. Defer unless a future REPS
  revision lowers the Tier 2 target.

Release ceremony

Standard pattern:

1. git tag -a v0.9.5 -m "Release v0.9.5 - Tier 2 perf optimisation (32x speedup)"
2. git push origin main
3. git push origin v0.9.5
4. cargo publish --dry-run --all-features
5. cargo publish
6. Confirm https://crates.io/crates/mod-alloc/0.9.5 is live.

GitHub release title: v0.9.5 — Tier 2 perf optimisation (~32x speedup).

---

Full Changelog: v0.9.4...v0.9.5

---

This discussion was created from the release v0.9.5 — Perf Optimization.