Fix GC.GetTotalMemory returning negative under regions GC

[kkokosa]

Fixes #106712.

Summary

GC.GetTotalMemory(false) intermittently returns a negative value under the regions GC (the default since .NET 7). The value originates in GCHeap::ApproxTotalBytesInUse, which estimates gen0 live bytes as gen0_size - gen0_frag using unsigned size_t arithmetic. gen0_frag (free-list + free-object space) is a per-generation total that spans every gen0 region, but gen0_size only summed region spans up to the ephemeral region. When gen0 retains another region (for example one pinned in place and swept rather than compacted), that region's fragmentation stays in gen0_frag while its span is dropped from gen0_size, so gen0_frag > gen0_size, the subtraction underflows, and the public API casts the near-2^64 size_t to a negative long.

In plain words: gen0 live bytes are "space minus holes." With segments, gen0 is a single contiguous block, so the holes are always a subset of the space and the subtraction can't go negative. With regions, gen0 is a linked list of blocks, and the code summed the holes over the whole list but measured the space over only part of it — so the holes could exceed the space and the unsigned subtraction wrapped.

The negative is only the visible symptom. The same miscount also makes GetTotalMemory silently under-report gen0 whenever the dropped span is smaller than gen0_frag (positive but too low) — observed returning ~0.4 MB where the correct value was ~16.0 MB.

Root cause

size_t gen0_frag = generation_free_list_space(gen0) + generation_free_obj_space(gen0);
// gen0_size summed region spans, but stopped at the ephemeral region:
//   if (gen0_seg == current_eph_seg) break;
totsize = gen0_size - gen0_frag;   // size_t underflow when gen0_frag > gen0_size

Two mechanisms, neither a GC race:

1. Multi-region gen0 span miscount (dominant). Any gen0 region linked after the ephemeral one had its span dropped from gen0_size while its fragmentation stayed in gen0_frag.
2. Discard-branch transient (secondary). In a_fit_free_list_p's free-list discard path, free_obj_space was incremented before free_list_space was decremented, so a lock-free reader could briefly observe both — a transient over-count.

Why a forced GC.GetTotalMemory(true) (and continued allocation) self-heals

The reported observation that a forced GetTotalMemory(true) returns a sane value and stops the negatives — and that continued allocation walks the value back positive — follows directly from Mechanism #1. The bytes are never wrong; only the lock-free measurement is.

• GetTotalMemory(true) forces blocking, compacting GC(s). Without pinning, gen0 is fully compacted: survivors are relocated, the swept-in-place regions linked after the ephemeral one are reclaimed, and gen0's free-space counters reset. Gen0 collapses to a clean layout where the counted span again covers all its fragmentation (gen0_frag ≤ gen0_size), so the next computation is correct — the bad layout is repaired.
• Continued allocation also recovers it without a GC. As the mutator allocates, alloc_allocated advances in the ephemeral region, so the counted gen0_size grows; once it climbs back above gen0_frag the subtraction stops underflowing. The value flip-flops as allocation and GCs change which snapshot a read catches.

The fix

• src/coreclr/gc/interface.cpp — walk every gen0 region when computing gen0_size (removing the early break at the ephemeral region), so the counted span matches the per-generation fragmentation total. This mirrors how generation_size() in plan_phase.cpp walks the whole region chain.
• src/coreclr/gc/allocation.cpp — reorder the discard-branch bookkeeping (unlink → free_list_space -= → free_obj_space +=) so the only transient a concurrent reader can observe is a harmless under-count instead of an over-count that could underflow. The final state is identical; this flips the transient's direction rather than removing it.

No clamping: the negative is fixed at its source (count all gen0 regions), not masked with a (gen0_size > gen0_frag) ? ... : 0 guard.

Why this is regions-only

With segments, gen0 lives inside a single contiguous ephemeral segment; its fragmentation is by construction a subset of the counted span, so the subtraction can never underflow. The bug requires gen0 to span multiple regions with one retained past the ephemeral one — only possible under regions. Empirically, with the same source and workload: built-in coreclr.dll (regions) → NEG −12,194,016 @ 97 ms; standalone clrgcexp.dll (regions) → NEG −12,158,984 @ 96 ms; standalone clrgc.dll (segments) → 0 negatives over 13.5 M probes. Matches the field report that DOTNET_GCName=clrgc.dll doesn't repro.

Reproduction

Two repros, both run by swapping only coreclr.dll between fixed and unfixed builds:

• Amplifier (reliable, used by the regression test). A console app keeps a large, continuously-refreshed ring of pinned tiny objects (forcing retained gen0 regions) while flooding gen0 with garbage and probing GC.GetTotalMemory on another thread. Fails in well under a second (~60–70 K probes, e.g. NEG −12,183,896), and fires even single-threaded (threads=1) — confirming a structural miscount, not a race.

• The reporter's exact repros from the issue (only change: a wall-clock cap), which use no pinning:

  @kg repro (unmodified logic)	Unfixed net11	Fixed net11
  Multithreaded (8 threads churning strings)	❌ NEG −2,883,184 after 140,958 probes @ 703 ms — "fails instantly", as reported	✅ 9.3 M probes / 25 s, no negative
  Single-threaded (rare)	did not fire in our windows (Debug 356 K / 900 s, Release 5.3 M / 900 s, min ≥ 0) — matches "takes longer to reproduce"	✅ clean

  Takeaway: pinning is a reliable amplifier, not a prerequisite — heavy multithreaded churn triggers frequent GCs that transiently reshuffle the gen0 region list into the same buggy layout.

Testing

New regression test

src/tests/GC/API/GC/GetTotalMemoryConcurrent.cs (+ .csproj), priority 1, process-isolated, GCStressIncompatible, [Fact] TestEntryPoint idiom. It runs the pinned + concurrent-probe workload and fails if any probe is negative. Pinning converts the rare transient into a deterministic, structural repro that fires in milliseconds.

Proven fail → pass (swapping only coreclr.dll fixed↔unfixed), on both Debug and full Release:

Build	Runtime	Result
Debug	Unfixed	❌ exit 101 — negative -11,997,832 after 100,285 probes
Debug	Fixed	✅ exit 100, 3/3 runs, ~2.8–2.9 M probes each, min ≈ +566,768
Release	Unfixed	❌ exit 101 — negative -11,970,520 after only 83,187 probes
Release	Fixed	✅ exit 100, 40.7–43.6 M probes, min ≈ +571,472

The Release run proves the fix holds on an optimized runtime; the unfixed Release binary reproduces the negative even faster than Debug.

Existing suite (no regressions)

GC/API/GC/GetTotalMemory, TotalMemory, TotalMemory2, GetGCMemoryInfo, GetTotalAllocatedBytes, GetAllocatedBytesForCurrentThread — all exit 100 against the fixed runtime (default regions GC).

The ".NET 10 fixed the single-threaded repro" claim — disproved

The issue notes the single-threaded (no-pin) repro was "fixed on .NET 10." Testing shows this is a measurement artifact, not a code fix: the accounting site is byte-identical since regions shipped in .NET 7 (git log -S on the guard block returns only PR #59283 plus mechanical gc.cpp → interface.cpp split commits — no .NET 8/9/10 change to the gen0 loop or subtraction). Against installed retail runtimes:

Repro variant	net 8.0.27	net 9.0.16	net 10.0.8	net 11 (unfixed)
pinned, single-threaded	NEG −12,141,504 @ 18 ms	NEG −11,967,528 @ 49 ms	NEG −12,103,120 @ 15 ms	NEG −12,189,472 @ 147 ms
no-pin, multithreaded	NEG −417,696 @ 50 ms	NEG −2,083,192 @ 10 ms	NEG −741,272 @ 25 ms	(regions repro)
no-pin, single-threaded	0 neg / 60 s	0 neg / 120 s	0 neg / 120 s	0 neg / 60 s

The structural underflow is single-thread-reproducible on every shipping version (15–49 ms with pinning). The no-pin single-threaded case is simply a rare transient on all versions (identical .NET 9 vs 10 behavior) — the perceived ".NET 10 fix" is timing noise, not a behavioral change. This PR is the first actual correction of the root cause.

Impact & blast radius

ApproxTotalBytesInUse is reached only via GCHeap::GetTotalBytesInUse, whose only callers are the System.GC.GetTotalMemory(bool) QCall (CoreCLR) and RhpGetTotalBytesInUse (NativeAOT). The one in-box managed consumer is RuntimeEventSource's gc-heap-size PollingCounter (GC.GetTotalMemory(false) / 1e6), which surfaces the negative through EventCounters / dotnet-counters / EventPipe / APM telemetry.

• No public API surface change — same signature and semantics, just a corrected number. Also repairs the forceFullCollection: true stabilization loop in GetTotalMemory(bool), whose diff = (newSize − size) / size convergence test was meaningless while size was underflowed.
• Risk is confined to accounting — neither change alters what the GC collects, promotes, or decommits. interface.cpp adds a bounded region walk (gen0 region count, single digits) under the already-held gc_lock; allocation.cpp only reorders two counter stores.

Performance impact

GetTotalMemory is a diagnostic API (the gc-heap-size counter polls it ~1×/sec), not a hot path. Local A/B (same build, only coreclr.dll swapped; no allocation in the timed section; batched quantum-free mean; workstation GC, x64):

Release (shipping configuration — authoritative):

Scenario	Unfixed	Fixed	Delta
Common case (pins=0, single gen0 region)	~45.4 ns	~43.7 ns	≈0 ns (identical result 626,712)
Fragmented (pins=40000, several regions)	~48.8 ns (under-counts: 0.4 M)	~47.0 ns (correct: 16.0 M)	≈0 ns (within noise)

Debug (checked; absolute numbers and delta both inflated by contract checks):

Scenario	Unfixed	Fixed	Delta
Common case (pins=0)	~535.7 ns	~537.8 ns	+~2 ns (≈0 %)
Fragmented (pins=40000)	~558.5 ns (under-counts: 0.4 M)	~611.2 ns (correct: 16.0 M)	+~53 ns (+~9 %)

The common case is free on either config (a single gen0 region means the fix walks the same one region). The Debug fragmented +53 ns is Debug-accessor overhead (contract checks on heap_segment_next/_mem/_allocated, in_range_for_segment), not a real cost — Release shows ≈0 in both scenarios. Cost is bounded by the (inherently small) gen0 region count.

Note

This pull request was prepared with the assistance of AI (GitHub Copilot). The root-cause analysis, fix, and regression test were reviewed by me before submitting.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
12 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @anicka-net, @dotnet/gc
See info in area-owners.md if you want to be subscribed.

[kkokosa]

@dotnet-policy-service agree company="Microsoft"

[Copilot]

Pull request overview

This PR fixes a correctness bug in CoreCLR GC accounting where GC.GetTotalMemory(false) could intermittently surface a negative long under regions GC, by ensuring gen0 span accounting covers all gen0 regions and by adjusting related free-space bookkeeping. It also adds a regression test intended to reproduce the negative-value symptom under concurrent allocation/pinning.

Changes:

• CoreCLR GC: In regions builds, compute gen0 size by walking all gen0 regions instead of stopping at the ephemeral region (GCHeap::ApproxTotalBytesInUse).
• CoreCLR GC allocation: Reorder discard-path counter updates to reduce transient fragmentation over-count exposure to concurrent readers.
• Tests: Add a new priority-1, process-isolated GC API regression test exercising concurrent pin churn + GC.GetTotalMemory(false) probing.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/coreclr/gc/interface.cpp	Fix gen0 size accounting under regions by walking the full gen0 region chain.
src/coreclr/gc/allocation.cpp	Reorder discard-path free-space counter updates to reduce transient over-count windows.
src/tests/GC/API/GC/GetTotalMemoryConcurrent.cs	New regression test that probes for negative GC.GetTotalMemory(false) while inducing retained gen0 regions via pin churn.
src/tests/GC/API/GC/GetTotalMemoryConcurrent.csproj	New test project configuration (process isolation, priority 1, GCStress incompatible).

[kkokosa]

Improved by 7ce70ab2ec7747a620650c64afd650a6cbfdd348

[kkokosa]

The only other place not under gc lock is a sibling allocator path for UOH (a_fit_free_list_uoh_p) but I would not touch it. This change was cosmetic change in the first place, just because it was easy. It is not for UOH path. @janvorli wdyt?

[janvorli]

I would leave out the reordering of the updates. It is not guaranteed to be seen in the code order e.g. on arm64 anyways and the compiler can reorder the instructions. Unless we want to add the barriers as @jkotas suggested.

[jkotas]

At minimum, the comment is misleading. The comment should make it clear that this is best effort and that GetTotalMemory may return inaccurate results.

[jkotas]

Hardware or C/C++ compiler can reorder the memory reads/writes. Does this need to be read/written with the right memory barriers (VolatileLoad/VolatileStore)?

[mangod9]

this is mostly by design where this accounting is best effort. Its a simple increment/decrement in most places like:

allocation.cpp#L1500
allocation.cpp#L1534

[jkotas]

It means that this change is just making the problem happen less often, not actually fixing it.

Also, if we want to keep this behavior, it should be documented. The current documentation for forceFullCollection does not say anything about a potential for returning grossly inaccurate result with forceFullCollection: false.

[kkokosa]

As explicitly said in the PR, this reordering is to occasionally under-count rather than over-count, but it does not solve inconsitency. Over-count is much more confusing. And imho it is not worth it to fight for it, docs mentions should work.

[janvorli]

It means that this change is just making the problem happen less often, not actually fixing it.

The main fix in this change is for the potentially massive miscounting we are getting with regions due to the not taking into account some regions for the space while using all them all for the holes size calculation.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[jkotas]

Suggested change

// (issue #106712).

Is the link to the issue required to understand the code?

[kkokosa]

Yeah, but tbh maybe the comment is too extensive in the first place?

[jkotas]

Right, AI likes to include a lot of fluff in the comments.