Skip to content

perf(gc): the copying minor traverses the young graph twice — the eligibility preflight is 22% of json_pipeline's hot phase #7645

Description

@proggeramlug

The copying minor traverses the whole young object graph twice: once in CopiedMinorEligibility::evaluate's preflight, to prove that nothing reachable is pinned, and again in the collector that copies. On json_pipeline the first traversal is 21.8% of the build_out phase and 14.5% of total wall, and it produces no collection result at all.

Measured (pinned quiet mini, current main v0.5.1363, canonical 500k fixture)

build_out at 500k is 1,956 ms, of which 1,872 ms is GC pause across exactly two collections. The minor is 1,123.5 ms, but its own phase_us.copying_nursery timer only accounts for 666.2 ms. The 457 ms residual is eligibility, and the symbolicated profile agrees to within 1%:

sample window = the build_out phase (marker-triggered, 2072 ms window vs a 2070 ms phase), 1504 leaf samples

  542  gc_collect_minor_copying_fast_path_with_eligibility   (the copy + RS work)
  294  CopyingNurseryPreflight::drain  <- mutable_root_preflight_reason
   40  CopyingNurseryPreflight::drain  <- dirty_slot_preflight_reason
  ---
  334 = 22.2% of build_out, ~460 ms

A probe that skips both preflight walks (PERRY_PROBE_SKIP_PREFLIGHT=1, an unmerged local arm), 5 interleaved rounds, output SHA identical on every row:

build_out 200k build_out 500k total 200k total 500k
main 734 ms 1,963 ms 1,158 ms 3,024 ms
preflight skipped 602 ms (−18.0%) 1,536 ms (−21.8%) 1,025 ms (−11.5%) 2,586 ms (−14.5%)

The subject was verifiably live and the decision verifiably unchanged (#7024/#7025): the minor still ran, still reported eligible=true, and still promoted 280,996,784 bytes — byte-identical to main — while the second traversal disappeared from the layout counters:

minor-cycle counter main preflight skipped
pause_us 1,121.3 ms 684.7 ms
layout_scans.pointer_slots_read 22,041,026 13,827,513
layout_scans.unknown_layout_slots_read 16,500,006 10,500,003
copying_nursery.promoted_bytes 280,996,784 280,996,784
sweep.freed_bytes 17,848 17,848

What the walk is actually deciding

mutable_root_preflight_reason and dirty_slot_preflight_reason do a transitive reachability walk (CopyingNurseryPreflight::drainscan_object_fieldscheck_ptr_with_reason, with a PtrHashSet seen set sized to the live young graph) to answer two booleans:

  1. is any transitively reachable Eden/FromSurvivor object GC_FLAG_PINNED? → PinnedYoungRoot / PinnedYoungDirtySlot
  2. was a non-arena candidate seen while the malloc registry is unavailable and was not empty at start? → MallocRegistryUnavailable

(2) is already an O(1) fact — CopyingPointerSet::malloc_registry_empty_at_start / malloc_registry_available.

(1) is O(live young graph) today, but its producers are countable. In production, GC_FLAG_PINNED is set at exactly three sites:

  • thread.rs:1490 — the cross-thread spawn promise (allocated in malloc space by construction, per the comment there)
  • thread.rs:1617 pin_promiseAtomics.waitAsync
  • string/format.rs:90 — the SMALL_INT_CACHE entries (longlived space, and check_ptr_with_reason only trips on Eden/FromSurvivor, so these can never cause the fallback)

The conservative scan does not use this bit — it pins into the separate CONS_PINNED set (barrier.rs:653), and it only runs at all when conservative_stack_scan_decision() == Scan, which the same traces show is skip_disabled for every automatic arena_bytes cycle. copying.rs:630 preserves the bit across a copy but never creates one.

Proposed shape

Route the three setters through one gc::pin_object(header) helper that also arms a process-wide monotone latch, and skip both preflight walks when the latch is clear and the malloc-registry question is already answered. Monotone means no unpin bookkeeping: a process that has ever pinned pays the walk forever, which is strictly no worse than today.

What this needs before it can merge — this is a safety check, not a cost

Removing a guard on the moving collector is the highest-severity change class in this tree, and an evacuated pinned object is a use-after-free for whoever holds the raw pointer (the cross-thread promise queue holds exactly that: a raw usize with no scanner). So:

  1. A scanning gate in lint that fails on any GC_FLAG_PINNED set outside gc::pin_object — the fix(runtime): JSX and raw-JSON shared a class id; add a scanning gate (#7587) #7589 class-id-collision gate is the precedent. Without it the latch's completeness is a comment, and the next pin site added silently makes the collector unsound.
  2. A sabotage test: plant a pinned Eden object, assert the copying minor still falls back with PinnedYoungRoot, and assert it goes green only because the latch was armed — i.e. delete the latch arming and watch the test go red (the perf(gc): break the survivor-promotion handoff livelock (#7592) #7594 lesson: a test asserting !preflight_reason(..) cannot fail on an empty heap).
  3. gc-ratchet, both arms on one host. layout_scans.* and malloc_validation_lookups change materially on every copying minor (the table above), so this needs a deliberate, documented counter shift rather than a silent one — same treatment as perf(gc): live-proportional collection budgets at both generations (#7592) #7596's 12_large_live_set.heap_total_bytes.

A cheaper alternative that avoids the pin proof entirely, if (1)–(3) prove too costly: hand the preflight's seen set and worklist to the copy phase as a pre-computed reachable set, so the graph is still walked once. Bigger refactor, no new invariant.

Found while re-deriving the next json_pipeline lever after #7624 and #7633 (#7592). The second-largest item on the same profile — the full collection's shadow BTreeSet valid-pointer census, 245.5 ms of a 748.3 ms full — is fixed separately and needs no new proof.

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceRuntime, compile-time, build-size, or memory performance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions