fix(gc): right-size idle arena capacity

[proggeramlug]

Summary

Right-size GC arena capacity after a burst leaves a persistently small live set. The idle reducer can now pay the arena's bounded two-full release debt without waiting forever for new mutator activity.

Changes

• Detect two consecutive post-collection samples at or below 50% arena utilization above a 32 MiB floor.
• Reuse the existing idle full/page-return path for at most two full observations, counting full observations already present in the low-utilization streak and stopping early at 60% utilization.
• Add 70% utilization and material-capacity-growth hysteresis so stable idle heaps cannot repeatedly buy full GCs.
• Export diagnostics for right-size episodes, starts, and released capacity, and document the policy alongside the idle reducer.
• Add policy boundary/hysteresis tests and an integration test proving one idle follow-up runs without mutator activity and no third full follows.

Related issue

Fixes #9709

Test plan

• cargo fmt --all -- --check
• cargo check -p perry-runtime
• RUST_TEST_THREADS=1 cargo test -p perry-runtime --lib (3,104 passed; 4 ignored)
• focused arena right-size and idle-reclaim tests
• scripts/check_test_registration.py
• scripts/check_gc_doc_claims.py
• scripts/check_gc_env_knobs.py
• scripts/check_docs.py
• scripts/test_affected_crates.sh --base upstream/main (runtime and 1,073/1,074 CLI tests passed; the sole CLI failure reproduces unchanged on the base commit because upstream PERRY_CONCAT_SITE_CACHE is absent from its build-cache inventory)
• scripts/run_lint_gates.sh (all 62 applicable static gates passed)
• scripts/run_gap_tests.sh with the pinned Node 26.5.1 oracle (full local snapshot comparison in progress; release compiler/runtime archives built successfully)
• Compile Claude Code 2.1.112 with the patched release runtime, then hold its static onboarding screen idle for five minutes under PERRY_GC_CENSUS
• Compile a 350,000-object burst/idle/burst probe: the cold burst took 39 ms and the second took 30 ms after arena capacity had been returned below the 32 MiB floor

The real-workload census observed arena capacity fall from 96.5 MB before the right-size episode to 36.7 MB, then 35.7 MB and 35.7 MB across the five-minute soak, with about 23 MB live. RSS fell from 401 MB at the first census to 132 MB at the last. Exactly one idle full was attributed to arena right-sizing.

Known upstream failures

• The self-test-checkers TLS job currently fails on five untouched files already present on main (fs/deferred.rs, gc/census.rs, gc/idle_compact.rs, gc/idle_reclaim.rs, and gc/oldgen_defrag.rs). Recent scheduled main runs fail the same gate; this PR's new TLS state uses crate::perry_thread_local! and is absent from the violation list.

• test_affected_crates.sh's sole CLI failure, codegen_env_vars_are_build_cache_inputs, reproduces unchanged on the base commit because PERRY_CONCAT_SITE_CACHE is missing from upstream's build-cache inventory. The affected runtime suite passes in full.

• Relevant release compiler/runtime archives build cleanly

• Added #[test] coverage in the affected runtime crate

• Updated docs/src/

Screenshots / output

[gc-idle-reclaim] start attempt=1 reason=arena_right_size ... arena_live=33865448 arena_capacity=96468992 right_size_fulls_remaining=2

75s:  live=23050960 capacity=36700160 rss=401432576
197s: live=23032704 capacity=35651584 rss=140521472
303s: live=23022496 capacity=35651584 rss=132104192

Checklist

• I have NOT bumped the workspace version or edited CLAUDE.md / CHANGELOG.md
• My commits follow the repository's commit prefix convention
• I've read CONTRIBUTING.md and agree to the Code of Conduct

Summary by CodeRabbit

• Bug Fixes

  • Improved idle-time memory reclamation so unused arena capacity from burst workloads can be returned more reliably.
  • Added safeguards to limit reclamation work and avoid repeated processing when utilization remains low.

• Documentation

  • Documented idle arena-capacity right-sizing behavior, thresholds, and reactivation conditions.
  • Added diagnostic counters for monitoring reclamation episodes, starts, and released capacity.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 5580fdf3-67fd-45e6-9602-15ce5b90f746

📥 Commits

Reviewing files that changed from the base of the PR and between e3618fc and bacadd4.

📒 Files selected for processing (11)

• changelog.d/9731-arena-right-sizing.md
• crates/perry-runtime/src/gc/arena_right_size.rs
• crates/perry-runtime/src/gc/copying.rs
• crates/perry-runtime/src/gc/cycle.rs
• crates/perry-runtime/src/gc/idle_reclaim.rs
• crates/perry-runtime/src/gc/mod.rs
• crates/perry-runtime/src/gc/policy.rs
• crates/perry-runtime/src/gc/tests/arena_right_size.rs
• crates/perry-runtime/src/gc/tests/idle_reclaim.rs
• crates/perry-runtime/src/gc/tests/mod.rs
• docs/src/internals/garbage-collector.md

Included review availability: Your plan provides up to 8 included reviews per hour; 5 remain after this review.

---

📝 Walkthrough

Walkthrough

The GC now tracks arena utilization across collections and opens bounded right-sizing episodes for idle heaps with sustained slack. Idle reclaim can service arena debt without mutator activity. Tests, diagnostics, changelog text, and internal documentation cover the new behavior.

Changes

Arena right-sizing

Layer / File(s)	Summary
Right-sizing policy and collection census crates/perry-runtime/src/gc/arena_right_size.rs, crates/perry-runtime/src/gc/policy.rs, crates/perry-runtime/src/gc/copying.rs, crates/perry-runtime/src/gc/cycle.rs, crates/perry-runtime/src/gc/mod.rs	The GC records full and minor collection observations. The new state machine triggers bounded right-sizing above a 32 MiB floor, stops at 60% utilization, and re-arms at 70% utilization or after material capacity growth.
Arena debt in idle reclaim crates/perry-runtime/src/gc/idle_reclaim.rs	Idle reclaim distinguishes activity starts from arena-debt starts. Arena debt can initiate owed full collections and appears in diagnostics.
Policy and idle-reclaim validation crates/perry-runtime/src/gc/tests/arena_right_size.rs, crates/perry-runtime/src/gc/tests/idle_reclaim.rs, crates/perry-runtime/src/gc/tests/mod.rs, changelog.d/9731-arena-right-sizing.md, docs/src/internals/garbage-collector.md	Tests cover trigger persistence, capacity floors, bounded episodes, early completion, hysteresis, and debt-driven follow-up collections. Documentation and the changelog describe the thresholds and lifecycle.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to bacad

Arena right-sizing is bounded by utilization, capacity, and idle-reclaim gates, with coverage for follow-up collection and re-arm behavior. No actionable current-head merge risk remains.

Sequence Diagram(s)

sequenceDiagram
  participant GC
  participant GC_policy
  participant arena_right_size
  participant idle_reclaim
  participant idle_reducer
  GC->>GC_policy: publish collection outcome
  GC_policy->>arena_right_size: record live bytes and full flag
  arena_right_size-->>idle_reclaim: report owed full collection
  idle_reclaim->>idle_reducer: start ArenaRightSize full collection
  idle_reducer->>arena_right_size: note_started()
  idle_reducer->>GC_policy: publish next collection census
  GC_policy->>arena_right_size: decrement debt or finish episode

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 44 functions across 9 files. (2 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the primary change: right-sizing idle GC arena capacity.
Description check	✅ Passed	The description includes the required sections, explains the implementation, references issue #9709, documents validation, records known upstream failures, and includes relevant output.
Linked Issues check	✅ Passed	The changes satisfy issue #9709 by adding arena right-sizing, reuse of the page-return path, utilization thresholds, hysteresis, a capacity floor, diagnostics, tests, and idle workload validation.
Out of Scope Changes check	✅ Passed	The changes are related to idle arena right-sizing and its validation. The runtime code, tests, documentation, diagnostics, and changelog entry support the linked issue objectives.

Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 44 functions across 9 files. (2 skipped: 2 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[proggeramlug]

Landed on main via merge train #9735 (rebase-merged, so your commits keep their authorship). Thanks!

[proggeramlug]

Follow-up: merge train #9735 captured the functional right-sizing commits but not the final hot-TLS conversion and decimal-unit label correction. Those two reviewed commits are isolated in #9736.