Why does Microsoft design .NET GC like this

[tonyqus]

I know there are a few blogs talking about how .NET GC works. But I'd like to dive deep and learn why it is designed like this. I also readed a few blogs from the GC Architect - Maoni Stephens. Maoni‘s blogs describe how GC works but not about how GC was designed.
For example, why only 3 gen instead of 4, 5, 6 gens? Why there is no way to solve LOH issue? And so on.

[EgorBo]

I'd recommend "Pro .NET Memory Management" book, I don't think your abstract question can be answered with just one reply here

[AustinWise]

I got nerd-snipped by the question "why not more than 3 generations?" and researched this topic a number of ways

Microsoft documentation

There are some articles articles on learn.microsoft.com about the garbage collector that address why there are generations and why there is a large object heap:

• Fundamentals of garbage collection
• The large object heap

They don't directly address why 3 and not some other number.

LLM

I asked Gemini Thinking "What is the optimally number of generations in a generational garbage collector?" and it did a pretty good job of discussing the trade offs. It included a "Why not more than three generations" section that says basically what I was planning on writing without having researched this topic:

While theoretically, more generations could provide even finer-grained control, the benefits often diminish quickly after 2 or 3 generations.

• Diminishing Returns: The weak generational hypothesis is typically strong enough that separating into 2-3 age groups captures most of the benefit. Adding more generations provides less and less marginal improvement in collection efficiency.
• Increased Complexity and Overhead: Managing and tracking too many generations adds significant complexity to the GC implementation and increases overhead without proportionally improving performance.
• Empirical Evidence: Practical experience in garbage-collected languages (like Java, .NET) shows that 2 or 3 generations are generally sufficient to achieve good performance for a wide range of applications.

Old Channel 9 videos

There is an Channel 9 video from 2009 with Patrick Dussud, the original author of the .NET GC. In it he mentions that one advantage of the GC is the objects in gen 0 tend to stay in the CPU cache, so by collecting just gen 0 and reusing the memory you get good cache locality.

My analysis of this is that if your generation sizing is related to CPU cache size, it does not make sense to have more generations than there are layers in the memory hierarchy. There are 4 levels of cache relevant to GC (L1, L2, L3, main memory). But it might not make sense to size gen 0 to be the same as the caches in a CPU. The sizing of the ephemeral segment in .NET is pretty big (tens to hundreads of megabytes), which is larger than modern CPUs last level cache. The Myths and realities: the performance impact of garbage collection paper had a benchmark showing that the optimal eden generation size in terms of reducing caches misses. Their conclusion was a size that was several times the last level cache of their CPU. Putting all of the above together, having two generations (young and old) should be sufficient to get the cache locality benefits of a GC.

There are two other videos on the site about the .NET GC that might be worth looking at for a historical perspective.

Research papers

As a starting point I searched for the most cited papers about garbage collection in the ACM digital library and traced through some of the references forward and backwards. I have not read everything, but I had a hard time finding a paper that specifically experimented with increasing the number of generations. These are the most interesting papers in my opinion:

• Myths and realities: the performance impact of garbage collection has some pretty good discussion on tradeoffs in garbage collector design and tuning.
• An adaptive tenuring policy for generation scavengers mentions both multiple generations and the idea for a separate heap for large objects.
• A real-time garbage collector based on the lifetimes of objects, one of the older papers about generational garbage collectors, proposes the idea of an arbitrary number of generations. Periodically a new generation is created and separately older generations are condemned.

Conclusion

Having two or three generations captures most the benefits of both cache locality and the weak generational hypothesis. Having more generations would increase implementation complexity and overhead.

[tonyqus]

Wow. Awesome!

You solved another question in my mind although I didn't express it directly here. I have a feeling that Maoni Stephens is not the original architect of GC. I'm not blaming her. She did a nice job blogging about .NET GC and explaining how it works.

Patrick Dussud, one of the CLR founders and chief architect of the .NET Garbage Collector, or GC, as developers call it. I wanted to learn about what a GC is, how it works, why it does what it does, how it will evolve, Patrick's history in the industry.

[tonyqus]

I will translate your answer to Chinese Simplified since I'm originally from the China mainland .NET community. And post a blog about this. Thank you!

[tonyqus]

I upload Patrick Dussud's C9 video to youtube. Youtube is much much better than C9.

https://www.youtube.com/watch?v=ki0jIyh0VHc