perf: opt-in jemalloc transparent huge pages for the store (closes #512)#582
Open
ELares wants to merge 1 commit into
Open
perf: opt-in jemalloc transparent huge pages for the store (closes #512)#582ELares wants to merge 1 commit into
ELares wants to merge 1 commit into
Conversation
At 1M+ random keys the per-shard hashbrown store tables (#570) and the value blobs are touched with poor locality, so each GET tends to miss the TLB (4 KiB pages mean many TLB misses on the -r 1M random-key hot path). Back the allocator's memory with 2 MiB transparent huge pages (THP) to cut the TLB-miss rate for the same coverage (estimated 3 to 10 percent, to be A/B measured on Linux stacked with the other cheap levers). Mechanism: jemalloc THP via malloc_conf, not per-allocation madvise. Both the store tables and the value blobs flow through the global allocator (jemalloc, ADR-0006), and hashbrown allocates its tables through that same allocator with no handle to madvise a specific region, so the cheapest effective lever is jemalloc's own thp:always boot option (it madvises MADV_HUGEPAGE on its extents), plus metadata_thp:auto for the arena metadata. This covers the tables and blobs process-wide with zero per-allocation code. Verified on Linux: a 256 MiB allocation under thp:always shows 274432 kB AnonHugePages in /proc/self/smaps, and the opt.thp mallctl reports "always". Linux gating: the thp: token is emitted into the compile-time malloc_conf only on target_os = "linux" (jemalloc compiles THP support on Linux and nowhere else). On macOS and other targets the string stays THP-free, so there is no jemalloc "Invalid conf pair" warning and the feature is inert; the macOS dev build still builds, boots, and serves. MSVC is unaffected (same cfg-gate as the allocator), and the musl static build is unaffected because malloc_conf is a compile-time static. Tunability ([[tunability-principle]]): THP is a behavior tradeoff, so it is a knob with a safe default, not a baked-in choice. It is DEFAULT-OFF (opt-in) because thp:always can raise RSS (2 MiB allocation granularity) and add khugepaged latency spikes on some kernels; keeping it off preserves an honest RSS figure for the maxmemory ceiling (ADR-0006). Two knobs, both default off: the build-time `hugepages` Cargo feature (flips the compiled default, Linux only), and the runtime override _RJEM_MALLOC_CONF=thp:always (or thp:never), which works on any shipped binary with no rebuild and overrides only the thp key. Documented in docs/design/CONFIG.md ("Transparent huge pages") and ADR-0006. Determinism (ADR-0003): THP is a memory-layout hint, not part of the engine decision path, and adds no clock or entropy, so it does not cross the determinism boundary. Tests: a main.rs unit test asserts the malloc_conf string carries thp:always iff Linux + the hugepages feature (locks in the gating and the opt-in default); a Linux integration test asserts jemalloc honors thp:always (opt.thp mallctl); and an ignored smaps probe (tests/hugepages.rs) verifies real AnonHugePages backing on a THP-capable host. Closes #512. Signed-off-by: Zeke <ezequiel.lares@outlook.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
perf-gate (A5)Same-runner ratchet of HEAD against the merge-base (both rebuilt and measured in this job).
Overall: PASS
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Dragonfly GET-gap perf (#507). At 1M+ random keys the per-slot hashbrown store tables (#570) + value blobs thrash the TLB (4KB pages). This enables jemalloc transparent huge pages (2MB) to cut TLB misses in exactly that regime.
Mechanism: jemalloc
malloc_confthp:always+metadata_thp:auto(not per-allocationmadvise-- hashbrown gives no handle to madvise a specific table, and both tables + blobs flow through the global allocator, so jemalloc's own THP option is the cheapest lever that covers everything process-wide). Applied inmain.rs's exported_rjem_malloc_confstatic, gated ontarget_os = "linux"+ thehugepagesfeature.Default OFF (opt-in), config-knobbed per the tunability tenet:
hugepagesCargo feature (Linux-only), and_RJEM_MALLOC_CONF=thp:always|thp:neverenv override (works on any shipped binary, no rebuild).Default-off because
thp:alwayscan raise RSS (2MB granularity) + add khugepaged latency spikes on some kernels -- which would inflate the very RSS themaxmemoryceiling accounts against. Documented tradeoff in CONFIG.md + ADR-0006. (jemalloc readsmalloc_confonce at init, so THP cannot be a liveCONFIG SETkey -- it is restart-required via the feature/env.)Verified huge pages are actually used: an integration test asserts jemalloc's
opt.thpmallctl returns"always"; a manual (ignored) smaps probe showed AnonHugePages 274432 kB backing a 256MiB allocation. Linux-gated (macOS/other get a THP-free string, no jemalloc warning); the musl static build is unaffected (compile-time malloc_conf).Tests
jemalloc_honors_thp_always_from_malloc_conf(feature on); default build unchanged; clippy-D warnings(default +hugepages) + io_uring build + invariant lints + fmt clean; no dashes.Closes #512. Part of #507.
🤖 Generated with Claude Code