Skip to content

Start mimalloc's scavenger thread explicitly at startup - #34502

Merged
dylan-conway merged 3 commits into
mainfrom
claude/mimalloc-explicit-scavenger-start
Jul 18, 2026
Merged

Start mimalloc's scavenger thread explicitly at startup#34502
dylan-conway merged 3 commits into
mainfrom
claude/mimalloc-explicit-scavenger-start

Conversation

@dylan-conway

Copy link
Copy Markdown
Member

What does this PR do?

mimalloc now leaves starting its background scavenger thread to the application instead of spawning it from a process-attach constructor (oven-sh/mimalloc#9). This bumps the mimalloc pin, binds the new mi_scavenger_start(), and calls it early in main — right after the signal setup, so that setup still runs single-threaded.

Without the call, purging quietly falls back to being allocation-driven and the event loop's heap handoff (mi_on_thread_idle_start) has no scavenger to hand off to; the explicit start keeps freed memory being returned to the OS off-thread as before.

How did you verify your code works?

The allocator side is covered by the mimalloc test suites (Debug + Release ctest, including the existing MIMALLOC_SCAVENGER=0 variant) in oven-sh/mimalloc#9. This side is a pin bump, an extern declaration mirroring the adjacent bindings, one call in main, and the corresponding process.versions.mimalloc expectation — build and tests via CI.

mimalloc now leaves starting its background scavenger thread to the
application rather than spawning it from a process-attach constructor.
Bump the mimalloc pin, bind the new mi_scavenger_start(), and call it early
in main so freed memory keeps being purged off-thread and the event loop's
heap handoff has a scavenger to hand off to.
@robobun

robobun commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator
Updated 5:31 PM PT - Jul 17th, 2026

@dylan-conway, your commit 733bfac is building: #74859

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 537eded5-c298-4de2-9a9e-c0a03878837a

📥 Commits

Reviewing files that changed from the base of the PR and between 1cb1f7a and 733bfac.

📒 Files selected for processing (2)
  • scripts/build/deps/mimalloc.ts
  • test/js/node/process/process.test.js

Walkthrough

The bundled mimalloc dependency is repinned to a new Git commit, and the corresponding process.versions test expectation is updated.

Changes

mimalloc revision update

Layer / File(s) Summary
mimalloc pin and version validation
scripts/build/deps/mimalloc.ts, test/js/node/process/process.test.js
Updates the fetched mimalloc commit and the expected commit hash in process.versions.

Possibly related PRs

Suggested reviewers: jarred-sumner, cirospaciari

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely states the main change: explicitly starting mimalloc's scavenger thread at startup.
Description check ✅ Passed The description matches the template and sufficiently explains the change and how it was verified.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Found 5 issues this PR may fix:

  1. Next.js standalone server.js under Bun 1.3.14: ~670MB idle RSS while live JS heap is only 14MB (Node 24 same build: 80MB) #34389 - ~670MB idle RSS while JS heap is only 14MB; classic symptom of mimalloc not purging freed arenas back to the OS
  2. Bun.build() leaks native memory per call (RSS unbounded, heapUsed flat) — kills long-lived watch/dev processes #34053 - Bun.build() RSS grows unboundedly while heapUsed stays flat -- freed native memory not returned to OS
  3. Native RSS grows linearly under sustained AWS SDK v3 Kinesis GetRecords; identical Node 22 workload is flat #30415 - Native RSS grows linearly under sustained workload while identical Node workload is flat
  4. Memory (RSS) in Bun Spawned Child Process Grows Slowly, Even When Idle #21560 - Child process RSS grows slowly even when idle -- no background scavenging to purge freed pages
  5. MIMALLOC_SHOW_STATS=1 environment variable does not work on Linux #28630 - MIMALLOC_SHOW_STATS=1 not working on Linux -- directly affected by mimalloc version bump

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #34389
Fixes #34053
Fixes #30415
Fixes #21560
Fixes #28630

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find any issues with the Bun-side wiring, but this bumps the process allocator pin and changes early main init ordering — worth a human look.

Checked: bun_alloc::mimalloc re-exports bun_mimalloc_sys::mimalloc, so the call path resolves; the new extern mirrors the adjacent mi_on_thread_idle binding (pub safe fn, no preconditions). Placement after the libc::signal block preserves that block's "before any other thread is spawned" SAFETY invariant, and the call is unconditional so ASAN/non-override builds (which still link and use mimalloc via MimallocArena/mi_heap_*) also get a scavenger. The process.versions.mimalloc test expectation matches the new pin.

Extended reasoning...

Overview

Four files: bumps the oven-sh/mimalloc commit pin in scripts/build/deps/mimalloc.ts, adds a pub safe fn mi_scavenger_start() extern in src/mimalloc_sys/mimalloc.rs, calls it once in src/bun_bin/lib.rs immediately after the SIGPIPE/SIGXFSZ setup, and updates the expected mimalloc hash in test/js/node/process/process.test.js.

Security risks

None identified. No user input is involved; the new FFI binding takes no arguments and is documented idempotent. The only ordering-sensitive concern is thread creation vs. signal setup, which is handled (see below).

Level of scrutiny

Higher than the diff size suggests. The Bun-side change is mechanical, but the substantive behavior change lives in the mimalloc fork commit being pinned (oven-sh/mimalloc#9), which is outside this diff. Bumping the process-wide allocator and inserting a thread-spawning call into main's init sequence is exactly the kind of change the repo's "Dependencies & vendoring" guidance flags for human review.

Other factors

  • Verified the call path: bun_alloc::mimalloc is pub use bun_mimalloc_sys::mimalloc (src/bun_alloc/lib.rs:23), so bun_alloc::mimalloc::mi_scavenger_start() resolves to the new extern.
  • The libc::signal block's SAFETY comment requires it to run "before any other thread is spawned"; placing mi_scavenger_start() immediately after keeps that invariant intact. bun_crash_handler::init() already precedes the signal block in existing code, so this PR doesn't change that relationship.
  • The call is not gated on #[cfg(not(bun_asan))]. That looks intentional: even when the #[global_allocator] is the system allocator under ASAN, mimalloc is still linked and used directly via MimallocArena / mi_heap_* / the Windows uv allocator, so the scavenger is still relevant there. The binding is declared safe with no preconditions, matching mi_on_thread_idle next to it.
  • On Windows the call now precedes uv_replace_allocator; the scavenger is internal to mimalloc and doesn't touch libuv, so that ordering is fine.
  • Deferring rather than approving because the allocator pin bump itself is the load-bearing part and warrants a maintainer's sign-off.

mimalloc keeps starting its scavenger during process init and now also
exports mi_scavenger_stop(), so bun needs no startup call after all; drop
the binding and the call, repin to the commit with the export, and keep the
process.versions expectation in step.
@dylan-conway
dylan-conway merged commit 8814dec into main Jul 18, 2026
5 of 38 checks passed
@dylan-conway
dylan-conway deleted the claude/mimalloc-explicit-scavenger-start branch July 18, 2026 00:34
Comment thread scripts/build/deps/mimalloc.ts Outdated
import type { Dependency, DirectBuild } from "../source.ts";

const MIMALLOC_COMMIT = "24211c6e7610ae7c4ec06040758ec90bd21a1c83";
const MIMALLOC_COMMIT = "475188181e4fc2816ec1caf3375a34fd9c0dd890";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The PR title and description still say this "binds the new mi_scavenger_start(), and calls it early in main", but commit 1cb1f7a ("Reduce to a plain mimalloc bump") removed both the binding and the startup call — the net diff is now just the mimalloc commit-hash bump and the matching process.versions expectation. Since the description becomes the permanent squash-commit message (per .claude/docs/landing-prs.md, PR process: "Re-sync title/description whenever review reworks the change"), the title/description should be updated to reflect that this is now a plain mimalloc pin bump before merging.

Extended reasoning...

What the finding is

The PR title is "Start mimalloc's scavenger thread explicitly at startup" and the description says it "bumps the mimalloc pin, binds the new mi_scavenger_start(), and calls it early in main — right after the signal setup". However, the second commit on this branch — 1cb1f7a "Reduce to a plain mimalloc bump" — removed both the mi_scavenger_start() FFI binding and the startup call, so the PR title and description no longer describe what the PR actually does.

Step-by-step proof

  1. Commit f82b780 ("Start mimalloc's scavenger thread explicitly at startup") added:
    • the mi_scavenger_start() extern declaration in src/mimalloc_sys/mimalloc.rs
    • a call to it in src/bun_bin/lib.rs after signal setup
    • the MIMALLOC_COMMIT bump in scripts/build/deps/mimalloc.ts
    • the process.versions.mimalloc expectation update in test/js/node/process/process.test.js
  2. Commit 1cb1f7a ("Reduce to a plain mimalloc bump") reverted the binding and the startup call. Its commit message says: "mimalloc keeps starting its scavenger during process init … so bun needs no startup call after all; drop the binding and the call."
  3. Net PR diff (git diff a2152850..HEAD --stat):
     scripts/build/deps/mimalloc.ts       | 2 +-
     test/js/node/process/process.test.js | 2 +-
     2 files changed, 2 insertions(+), 2 deletions(-)
    
    src/bun_bin/lib.rs and src/mimalloc_sys/mimalloc.rs are not touched.
  4. git grep mi_scavenger_start src/ returns no matches — the binding is gone from the tree.

So the delivered change is exactly: bump the pinned mimalloc commit from 24211c6… to 4751881… and update the corresponding test expectation. Nothing about scavenger startup is changed on the Bun side; per the 1cb1f7a commit message, mimalloc at the new pin still auto-starts its scavenger during process init.

Why this matters

The repo's own PR-process rules (.claude/docs/landing-prs.md, referenced from REVIEW.md) state:

The PR description is the permanent squash-commit message — keep it true. … Re-sync title/description whenever review reworks the change.

If this merges as-is, the squash commit on main will be titled "Start mimalloc's scavenger thread explicitly at startup" and its body will describe binding mi_scavenger_start() and calling it in main — none of which is present in the merged diff. Anyone later bisecting a scavenger-related regression, or reading git log to understand when/why explicit scavenger startup was added, would be misled: this commit does not add an explicit startup call. The description's second paragraph ("Without the call, purging quietly falls back to being allocation-driven…") is also now factually wrong for this PR, since there is no call and — per the 1cb1f7a message — none is needed.

What existing safeguards missed

CodeRabbit's "Title check" and "Description check" both passed, but they were run against commit f82b780 — before 1cb1f7a landed and dropped the change the title/description describe. Nothing re-validated the metadata after the rework commit.

Impact

No runtime, correctness, or build impact — the code that ships is fine. The only consequence is a misleading permanent squash-commit message in main's history. That's why this is filed as a nit, not a merge blocker.

Fix

Retitle to something like "Bump mimalloc to 4751881" (or similar) and rewrite the description to say what the net diff does: bump the pinned oven-sh/mimalloc commit and update the process.versions.mimalloc test expectation. If it's useful context, note that the new pin keeps the scavenger auto-starting during process init so no Bun-side change is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants