[Feature Request]: CoreCLR GC cache topology awareness for Hybrid CPUs (P/E-Cores) and upcoming bLLC architectures (Intel Nova Lake)

[ahmetenesay]

Problem Statement / Motivation

With the rise of hybrid CPU architectures (P-cores / E-cores), managing Garbage Collector (GC) thread affinity and memory layout has become increasingly complex.

Currently, attempts to pin GC execution (or background GC threads) strictly to E-cores to save power or keep P-cores free for game/rendering loops often lead to system instability, thread starvation, or severe latency spikes.

While modern E-cores (e.g., Crestmont, Skymont) possess strong raw IPC capabilities, their primary bottleneck during heavy GC sweep/compact phases is memory/cache bandwidth. E-cores typically share smaller L2 caches per cluster and have limited L3 cache allocation slices. When GC threads perform massive memory traversals on E-cores, cache thrashing occurs, choking the runtime.

Furthermore, upcoming hardware architectures—such as Intel's Nova Lake (400S series)—are set to introduce bLLC (Big Last Level Cache) / dedicated cache topologies. CoreCLR needs ahead-of-time architectural preparation to adapt its GC allocation, marking, and thread scheduling strategies to these heterogeneous cache structures rather than treating all logical cores or cache lines uniformly.

---

Proposed Behavior / Feature

We propose enhancing the CoreCLR GC engine with Dynamic Cache & Hybrid Core Topology Awareness:

1. Cache-Aware GC Thread Scheduling:

   • Instead of naive core pinning or treating all logical processors as homogeneous, GC dynamic thread scheduling should consider L2/L3 cache boundaries per core type.
   • Heavy GC phases (e.g., Mark/Compact) should dynamically balance work across P-cores when cache pressure on E-core clusters exceeds a threshold, preventing system instability and latency spikes.

2. Preparation for bLLC (Nova Lake) & Large Cache Topologies:

   • Optimize GC heap segment layout and mark-stack traversals to leverage large side-caches/bLLC without invalidating L1/L2 caches of execution-critical P-cores.
   • Provide better heuristic integration with Windows/Linux schedulers (e.g., Thread Director) to inform the OS about GC thread memory intensity vs. compute intensity.

---

Alternative Solutions Considered

• Manual OS-level Affinity Pinning: Forcing GC threads to specific cores via environment variables or OS APIs (SetThreadAffinityMask). Drawback: Leads to cache thrashing and lock-contention instability on modern E-core clusters due to lack of internal GC state awareness.
• Disabling E-cores entirely: Drawback: Sacrifices multi-threaded throughput and efficiency on high-core-count consumer desktop CPUs.

---

Additional Context

• Target Area: area-GC
• Impact: High-throughput desktop applications, C# game engines, and real-time server workloads running on Intel Alder Lake, Arrow Lake, and future Nova Lake platforms.

[dotnet-policy-service]

Tagging subscribers to this area: @anicka-net, @dotnet/gc
See info in area-owners.md if you want to be subscribed.

[huoyaoyuan]

Currently, attempts to pin GC execution (or background GC threads) strictly to E-cores to save power or keep P-cores free for game/rendering loops often lead to system instability, thread starvation, or severe latency spikes.

Do you have evidence or measurement for how it matters in real world? It's not recommended to pin any thread based on stereotype, without real world measurement. Instead, we should rely on OS scheduler as possible.

• Instead of naive core pinning or treating all logical processors as homogeneous, GC dynamic thread scheduling should consider L2/L3 cache boundaries per core type.
• Heavy GC phases (e.g., Mark/Compact) should dynamically balance work across P-cores when cache pressure on E-core clusters exceeds a threshold, preventing system instability and latency spikes.

This is something we would explicitly avoid. The CLR runtime is mostly unaware of CPU cache, leveraging underlying OS and CPU to provide optimal scheduling. The only awareness about cache is at cache line level (avoid tampering the same cache line too much).

Preparation for bLLC (Nova Lake) & Large Cache Topologies:

• Optimize GC heap segment layout and mark-stack traversals to leverage large side-caches/bLLC without invalidating L1/L2 caches of execution-critical P-cores.

How can this make a difference? The Last Level Cache is shared by all CPU cores. It's capacity would not affect inter-core synchronization.

[MichalPetryka]

The CLR runtime is mostly unaware of CPU cache, leveraging underlying OS and CPU to provide optimal scheduling

I'm pretty sure the GC does explicitly query CPU cache sizes for heuristics.

[tannergooding]

Instead, we should rely on OS scheduler as possible.

Notably this is where the modern "weak-affinitization" APIs often come into play and hardware manufacturers (and OS vendors) often have explicit guidance around them: https://www.intel.com/content/www/us/en/developer/articles/guide/12th-gen-intel-core-processor-gamedev-guide.html

You do often want to default to the OS scheduler and you almost never want to strongly affinitize. But you can assist the OS by providing weak affinitization hints instead.

This is something we would explicitly avoid. The CLR runtime is mostly unaware of CPU cache, leveraging underlying OS and CPU to provide optimal scheduling. The only awareness about cache is at cache line level (avoid tampering the same cache line too much).

Petris is right and this is rather something explicitly queried by the GC already. It is also something that some of the higher end perf oriented algorithms are mindful of, particularly with regards to when to start using non-temporal loads/stores

How can this make a difference? The Last Level Cache is shared by all CPU cores. It's capacity would not affect inter-core synchronization.

This is not strictly true. While the last level of cache is ultimately accessible to all cores, its not in the way many people assume.

For example, while a modern AMD Ryzen 7950X has 64MB L3 cache and 16 cores/32 threads, this is split by CCX, so you get 8 cores/16 threads with 32 MB of access. The other 32 MB is then technically accessible, but not really directly and at a significantly higher latency, so its almost better to treat it like some L3.5 instead (and can in fact be slower than a proper RAM access). Further, this is then not truly the same speed even within a single CCX. Due to sharing and the 16-way associativity you don't strictly want to treat it as "I have 32MB accessible". You rather really want to treat it closer to 16KB L1, 512KB L2, 2MB L3 per thread (or 32KB L1, 1MB L2, 4MB L3 per core)

It actually gets fairly complex in how you handle/consider all these aspects and there is likely some amount of data/topology that is beneficial to surface to apps desiring it longer term. That being said, this is often very hardware specific and we expose the relevant intrinsics on xarch to query it. The underlying OS APIs provide the best way to query the info otherwise (as on Arm64 most such data requires Ring 0 to be queried) and so at best any such data surfaced via a new managed API would be for convenience and likely not comprehensive.

---

This request is largely not actionable as is and is placing all the design work and thought onto the dotnet team. While we're of course capable of doing that design work, it has to be prioritized relative to other requests, features, bug fixes, etc. Your best bet to get something like this to move forward is to create a concrete API proposal for the data that is desired to be surfaced here. I would speculate it belongs off Environment, RuntimeHelpers, or some surface off System.Threading.

[ahmetenesay]

This might seem unnecessary at first glance, but when writing a game engine, I'm using hints to lock the main loop onto the P-Cores in the OS Scheduler, but GC loops can contaminate the P-Cores, which degrades the main application's performance. Please reconsider.

[huoyaoyuan]

This might seem unnecessary at first glance, but when writing a game engine, I'm using hints to lock the main loop onto the P-Cores in the OS Scheduler, but GC loops can contaminate the P-Cores, which degrades the main application's performance.

There are benefits I can see to run GC on P-Cores:

• By running GC on the same core of other threads, it naturally shares the cache used by the threads
• As discussed above, GC is more memory intensive, and P-Cores are typically having better cache performance

Do you have any real world data for comparison? By deferring to OS scheduler totally, or affinatize them to different cores.

Different kind of workloads are having different memory characteristics and expectations. The runtime should be open and flexible, but may not be specifially tuned for certain workload.

[ahmetenesay]

I don't have precise numerical data, but I do have observational data. In my game engine, I pinned the GC to E-Cores absolutely because my goal is clear and precise: to dedicate the P-Cores to the pure main game loop. At the time, only the JetRains Rider IDE was open, then the E-Cores suddenly filled up and the system became unstable; the FPS dropped! Like a horror game, right? 😆 Huh! When I checked in Task Manager, the E-Cores were stuck. Here's my system:
CPU
Intel(R) Core(TM) Ultra 5 245K

Base speed: 4.20 GHz
Slots: 1
Cores: 14
Logical processors: 14
Virtualization: Enabled
L1 cache: 1.4 MB
L2 cache: 26.0 MB
L3 cache: 24.0 MB

Usage N/A% (I intentionally wrote N/A)
Speed ​​5.0 GHz
Runtime 0:09:16:10
Processes 506
Threads 10025
Identifiers 251340
(I copied these values ​​right now, meaning the PC is currently idle)

GPU 0
NVIDIA GeForce RTX 3060

Driver version: 32.0.16.1047
Driver date: 19.05.2026
DirectX version: 12 (FL 12.2)
Physical location: PCI bus 1, device 0, function 0

Usage 1%
Dedicated GPU memory 1.8/12.0 GB
Shared GPU memory 0.1/17.9 GB
GPU Memory 1.9/29.9 GB

Memory

32.0 GB DDR5

Speed: 6000 MT/s
Slots used: 2 / 2
Form factor: DIMM
Hardware dedicated: 664 MB

Available 13.5 GB
Cached 11.6 GB
Completed 33.0/46.8 GB
Paging pool 1.1 GB
Non-paging pool 1.4 GB
Used (Compressed) 17.8 GB (1.2 GB)

iGPU and NPU are present, but shouldn't they be relevant to the topic? If I had to say it, the Unified Video Memory is 18 GB, 18 GB for each unit (dGPU-iGPU-NPU).

[ahmetenesay]

I'm painfully aware that thread pinning is flawed, but I also complain about background jobs like GC occupying P-Cores! I think this is a universal problem with VMs?

[ahmetenesay]

Perhaps GCs should be completely overhauled, focusing on E-Core processing despite their heavy processing loads...

[ahmetenesay]

Also, there might have been a misunderstanding regarding my mention of bLLC: Intel's Skymont-based efficiency cores, released with Arrow Lake (Ultra 200S), delivered surprisingly high IPC values, surpassing the performance cores of previous generations! However, these naturally have very little share in L2 and L3 cache, which explains why the high IPC, contrary to my intuition, didn't translate into GC-friendly performance. This forces me to accept that the E-Core-focused GC architecture I mentioned in my previous answer isn't feasible as I originally framed it.

However, according to public roundups on Nova Lake (Ultra 400S), the picture changes significantly: on top of a traditional L2+L3 of 160-320 MB (vs. 76 MB on Arrow Lake), Intel is adding a separate bLLC (Big Last-Level Cache) layer of 144-288 MB depending on SKU — and notably, unlike AMD's 3D V-Cache which sits asymmetrically on a single compute tile, Intel's bLLC is symmetric across both tiles specifically to simplify OS/runtime scheduling, per their own stated design goal.

Given that much larger and more evenly distributed cache budget, I'd expect E-core clusters to have meaningfully more headroom for memory-bound work like GC sweep/compact phases. If that turns out to be the case, how would you approach this — would it change your position on cache-topology-aware scheduling, or do you still see it as out of scope for the runtime regardless of the underlying hardware?

[huoyaoyuan]

In my game engine, I pinned the GC to E-Cores absolutely because my goal is clear and precise: to dedicate the P-Cores to the pure main game loop.

Do you have profiling data or architecture analysis to support the goal being beneficial at all? Or is it simply a guess about core arrangement?