v0.9.1 — Inline Backtrace Capture

[jamesgober]

mod-alloc v0.9.1 — Inline Backtrace Capture

Date: 2026-05-14
Compare: v0.9.0...v0.9.1

Headline

Tier 2 lands. With --features backtraces and frame pointers
enabled, every tracked allocation captures up to 8 frames of its
call site via inline frame-pointer walking. Per-call-site
aggregation is exposed through a new ModAlloc::call_sites()
API. Zero external dependencies — no backtrace crate, no
addr2line, no gimli. Symbolication ships in v0.9.2.

What's new

Public API additions

• unsafe-free hot path additions to GlobalAlloc. alloc,
  alloc_zeroed, and realloc now invoke
  backtrace::record_event after the existing counter update
  when the backtraces feature is on. dealloc does not capture
  (matches dhat: call sites describe who allocated, not who
  freed).
• ModAlloc::call_sites() -> Vec<CallSiteStats> drains the
  per-call-site aggregation table into a vector for inspection
  or reporting. Behind #[cfg(feature = "backtraces")].
• CallSiteStats public type carrying frames: [u64; 8],
  frame_count: u8, count: u64, total_bytes: u64. Frames are
  raw return addresses; symbolication lands in v0.9.2.

Capture mechanism

• Inline core::arch::asm! reads the FP register (rbp on
  x86_64, x29 on aarch64). No extern "C" stubs, no build.rs
  code generation, no backtrace crate. Stable since Rust 1.59,
  well under MSRV 1.75.
• Pure-Rust FP walk with five mandatory hardening checks per
  iteration (null, alignment, in-range, monotonicity, max-frame
  cap). The walker is total: for any input bytes it terminates
  within 8 iterations without UB. Proven via SplitMix64-driven
  random-workload fuzz (10,000 iterations) in
  tests/backtrace_fuzz.rs.
• Per-thread stack-bounds discovery queries the OS once per
  thread, caches in TLS. GetCurrentThreadStackLimits on
  Windows, pthread_getattr_np on Linux,
  pthread_get_stackaddr_np on Darwin / *BSD. All inline
  extern "C" / extern "system"; no libc crate.

Storage architecture

• Per-thread arena. 64 KB OS-page region per thread (via
  VirtualAlloc on Windows, mmap on POSIX), 512 events per
  flush. Flushed synchronously when full, plus on thread exit via
  a TLS Drop impl.
• Global aggregation table. Fixed-size, open-addressed,
  atomic-only hash table. Default 4,096 buckets × 96 bytes ≈
  384 KB. Allocated lazily once via raw OS pages on the first
  event recorded. Override via MOD_ALLOC_BUCKETS env var at
  process start (clamped [64, 1_048_576], rounded up to next
  power of two).
• Inline FxHash variant for the frame-array hash. Eight words
  of input, two ops per word, no memory traffic. Hash value 0 is
  remapped to 1 so the bucket's atomic hash field can use 0 as
  the "empty" sentinel.

Reentrancy safety

The walker reads memory only inside the cached stack bounds and
only at 16-byte-aligned addresses, which avoids page faults by
construction. The existing IN_ALLOC reentrancy guard from
v0.9.0 covers any pathological allocation triggered
transitively from inside the backtrace path (e.g. libc lazy-init
during the very first pthread_getattr_np). No new mechanism
introduced.

Concurrency

Each bucket uses a two-phase publish protocol: CAS on hash
first (Release ordering) to claim the slot, then plain stores of
sample_frames, then frame_count.store with Release to mark
the bucket fully populated. Readers gate on frame_count > 0
after observing a non-zero hash; this prevents torn reads of the
sample frames. Validated by the 32-thread aggregation stress
test (tests/backtrace_concurrent.rs).

Test additions

• src/backtrace/* unit tests: hash determinism + clamping +
  collision floor, walker null / alignment / range / monotonic /
  max-frame stops, arena round-trip, table single-site and
  multi-site aggregation, raw-mem alloc / free, stack-bounds
  discovery sanity.
• tests/backtrace_real_chain.rs — captures from a chain of
  #[inline(never)] functions ending in Vec::with_capacity.
• tests/backtrace_fuzz.rs — SplitMix64 PRNG drives 10,000
  randomly-sized allocations + occasional drop chains and
  format-heavy paths. Verifies the walker is total.
• tests/backtrace_concurrent.rs — 32 threads × 1,000 allocations
  across distinct call paths. Verifies the table claim races
  resolve without deadlock or drop.

Example

• examples/backtraces.rs — installs ModAlloc as
  #[global_allocator], runs three known call paths
  (alloc_small / alloc_medium / alloc_large), and prints
  the top sites by total_bytes.

Tooling

• build.rs (one-off approved exception per
  .dev/DIRECTIVES.md section 2.1). Detects whether RUSTFLAGS
  includes -C force-frame-pointers=yes when the backtraces
  feature is on and emits cargo:warning= if not. Never fails
  the build; the walker degrades gracefully.
• .cargo/config.toml in the crate root enables frame
  pointers for the crate's own builds (tests, examples,
  doctests). Downstream consumers must set the flag in their own
  build.
• CI ASAN nightly job runs the test suite under
  -Zsanitizer=address with -Zbuild-std on Linux x86_64.
  Cheap insurance for the unsafe walker path.

Design notes

Per-thread arena is now load-bearing

v0.9.0 deferred per-thread buffering because four u64
counters do not benefit from amortising indirection. v0.9.1
captures up to 80 bytes per event (8 × 8-byte frames + size +
frame_count). At that size, going through a global lock-free
queue per allocation would serialise the hot path. The per-
thread arena absorbs 512 events between flushes, amortising the
hash + table-insert cost across the batch.

Why VirtualAlloc / mmap instead of Vec

The backtrace path runs inside the alloc hook. Allocating its
own state through ModAlloc::alloc would recurse (the existing
guard catches it, but the storage would silently bypass tracking
and we want clean accounting). Raw OS pages avoid this entirely.

Capture-on-realloc matches dhat

Per maintainer guidance, realloc captures all events including
shrinks. This matches dhat's per-event accounting and preserves
v0.9.3's dhat-compat compatibility. Documented in the rustdoc.

Configuration is runtime, not build-time

MOD_ALLOC_BUCKETS is read at runtime via std::env::var
inside the global table init path. This is intentional: a
build-time constant would force users to recompile to change the
table size, and a build.rs doing env-var reads would conflict
with our zero-build-script policy. The runtime read happens once
per process under the reentrancy guard.

Migration

Default builds are unchanged. The counters Tier 1 path is
identical to v0.9.0. No code edits needed.

Users opting in to backtraces:

[dependencies]
mod-alloc = { version = "0.9", features = ["backtraces"] }

# .cargo/config.toml
[build]
rustflags = ["-C", "force-frame-pointers=yes"]

Without the rustflag, build.rs emits a cargo:warning= and the
walker captures zero or one frame at runtime.

Verification

Full matrix run on Windows host (x86_64):

• cargo build (default features)
• cargo build --no-default-features
• cargo build --features counters
• cargo build --features backtraces
• cargo build --features dhat-compat (still no-op as documented)
• cargo build --all-features
• cargo +1.75 build --all-features (MSRV)
• cargo fmt --all -- --check
• cargo clippy --all-targets --all-features -- -D warnings
• cargo test --all-features: 54/54 pass (32 unit + 4
  v0.9.1 integration + 4 v0.9.0 integration + 5 smoke + 9
  doctests)
• Banned-word scan (comprehensive, robust, seamless,
  leverage): 0 hits across shipping files
• Em-dash scan: 0 hits across shipping files

CI matrix unchanged set (ubuntu-latest, macos-latest,
windows-latest) plus new ASAN nightly Linux job.

Measured overhead

cargo run --release --features backtraces --example bench_overhead
(1,000,000 × 64-byte alloc / dealloc cycle):

Build	Per cycle
v0.9.0 baseline (counters only)	34.9 ns
v0.9.1 with backtraces	1,915 ns

Net backtrace overhead: ~1,880 ns per allocation. REPS
section 6 sets the Tier 2 target at <200 ns. We are roughly 10x
over.

Likely contributors (not investigated in this milestone):

• read_volatile in the walker prevents reordering — intentional
  for the safety contract, but costly.
• thread_local! try_with closure overhead inside IN_ALLOC
  and the arena slot access.
• Atomic-store loop in arena append (writes 80 bytes as 10
  AtomicU64 stores rather than a memcpy).
• Spin-wait in table::wait_published even when the bucket has
  long since been published.

The arithmetic suggests several of these are individually
optimisable; the headline figure is honest end-to-end including
all kernel/OS noise. Treated as a known limitation; an
optimisation pass is filed as a separate milestone (see
"Follow-ups" below) so this release can ship and unblock
downstream work.

Capture quality on stock builds

Default Rust release builds compile std without frame pointers.
The walker correctly stops at the first FP-less frame rather than
crashing, which on a typical workload produces 1-frame traces
(the immediate caller of GlobalAlloc::alloc inside std's
allocation infrastructure).

For deep, application-meaningful traces, downstream users need
either:

• cargo +nightly -Z build-std=std with
  -C force-frame-pointers=yes (rebuild std with FPs), or
• A toolchain image built with FPs enabled application-wide.

v0.9.2's symbolication work may explore frame-pointer-less
unwinding (SEH on Windows, DWARF .eh_frame on POSIX) but that is
a separate scoping decision.

Limitations

• Performance is 10x over the REPS Tier 2 target. See
  "Measured overhead" above. Functional correctness verified;
  micro-optimisation deferred.
• Backtraces are shallow without an FP-built std. Documented
  above and in the rustdoc.
• Hash collisions are not detected. Two distinct call sites
  with identical 64-bit FxHash values get conflated in the
  aggregation table. Statistically rare; not a correctness issue
  but a visibility one.
• Table-full events are silently dropped. When the open-
  addressed table reaches full load (4,096 distinct sites by
  default), new sites are dropped. A dropped_entries diagnostic
  counter is planned for v0.9.2's Diagnostics API.
• TLS destructor coverage. The per-thread arena's flush-on-
  exit relies on thread_local! destructor invocation. On abort
  (std::process::exit), destructors do not run and the last
  N events on each thread remain unflushed. Acceptable; the
  global table holds the bulk.

Downstream impact

dev-bench MSRV unblock: still in progress. v0.9.1 does not
complete the dhat replacement (DHAT-format output is v0.9.3),
but it ships the capture mechanism dev-bench will eventually
read from. The overhead figure above means dev-bench should
not enable backtraces for every benchmark run; gating it
behind a bench-with-traces profile keeps the default fast.

Strategy context:
C:\Dev\libraries\Rust\_strategy\MSRV_AUDIT_DEV_LIBS.md.

Follow-ups

• Optimisation pass to close the 10x perf gap. Candidates
  listed in "Measured overhead". Suggested as a v0.9.1.1 patch
  release or folded into v0.9.2's symbolication work.
• Diagnostics API (Vec<DroppedEntry>, table-full counter,
  per-bucket fill rate) deferred to v0.9.2.
• Symbolication is v0.9.2's entire scope.

Release ceremony

Standard pattern:

1. git tag -a v0.9.1 -m "Release v0.9.1 - inline backtrace capture"
2. git push origin main
3. git push origin v0.9.1
4. cargo publish --dry-run --all-features
5. cargo publish
6. Confirm https://crates.io/crates/mod-alloc/0.9.1 is live.

GitHub release title: v0.9.1 — Inline Backtrace Capture.
Tag as pre-release; 1.0.0 waits for v0.9.4 (dev-bench
integration) to ship and bake.

---

Full Changelog: v0.9.0...v0.9.1

---

This discussion was created from the release v0.9.1 — Inline Backtrace Capture.