Auto-Compact LOH on OutOfMemory Exception, or other trigger

[najak3d]

This question ties to this post:
#47659

which references this partial fix:
https://docs.microsoft.com/en-us/dotnet/core/runtime-config/garbage-collector#heap-limit-percent

But leaves much to be desired, as the result is tedious/non-elegant code to ensure you never get an OOM (or ASP restart when memory exceeds a limit).

What I would suggest is that you add a GC setting like this:

AutoCompactLargeObjectHeapOnOutOfMemoryException = true

If 'true', then when an OutOfMemory exception occurs (OOM), it would do a GC2 and Compact the LOH, and then would RETRY the allocation that failed!

How useful would this be?!?!? We can have 1000's of places in our code that might make the "final allocation that causes the OOM" - and we don't want to have 1000 mechanisms for "try allocation, catch OOM, Compact LOH, then retry allocation". It would be FAR more elegant if the .NET runtime would simply do this for us!... then if after LOH Compaction, and OOM occurs again, then it would actually throw the exception, and maybe crash your app.

Otherwise, we're forced to "over compensate" and have many forced Compaction of LOH, "just-in-case". And that is so non-elegant and not nearly as effective as simply having .NET runtime do this for us.

Essentially, this feature would automatically change many CRITICAL exceptions, into a simple "few second hiccup" at worst, while alleviating much stress from memory-intensive .NET apps.

Is this something the .NET runtime could feasibly do for us?

[danmoseley]

cc @mangod9

[najak3d]

Using DiskCache would actually be a FINE solution, so long as it notifies us of the condition (give us an event to register for) -- and then we'll MANUALLY force LOH compaction, which should then pull that allocation from the LOH. In this instance, the hiccup that results would maybe be an extra second, added by the write/read to/from disk cache for the one allocation.

So if automating the "catching OOM, compacting LOH, then retrying the allocation", then maybe using DiskCache as an alternative to the OOM -- and notify us, so that we can manually handle this -- that would also be 10000% better than the current paranoia that all memory intensive .NET apps face now, without a good alternative.

[msedi]

@najak3d: I've tried the diskcache with memory mapped files. The problem is that in my situation I have two concurrent processes: 1. my process the produces/creates memory. 2. the memory manager. I'm producing obviously too much memory in a very short time so that the MM is not able to follow and I finally get in the same condition as coming in low memory LOH conditions and even worse the MM fills the whole physical memory, then enters the page file and ultimately halts the whole system.

Finally I had to the conclusion theat also MM-files are not sufficient and I had to include my own throttling to given the MM enough time to page the data. But even then I had no possibility to find out that I'm back in good memory conditions again. So I had to call WinAPI methods (FlushFileBuffers and EmptyWorkingSet) to get back to a good condition.

I currently left trying MM-Files because I had no good control over the paging system, too much WinAPI was involved and I wanted to be platform independent. So I currently created my own memory management for this purpose. It works quite well but still having my own "cleanup thread" running that continously checks memory conditions and performs a cleanup (removing the LOH objects from the list). After the cleanup I have to run the well known LOH compaction.

Regarding the OOM have you tried the MemoryFailPoints. I did them one but I seemed to a bit out and was also slowing down things because it fails awkward allocating an object to catch the exception if the memory could be allocated.

[najak3d]

Thanks for chiming in @msedi. That helps Microsoft gurus to understand the severity of this issue in .NET, and the utter sweetness and power of providing a built-in solution.

I wasn't suggesting a system "reliant upon MM files" (disk cache), but rather to allow us to configure .NET to resort to a MM file whenever it hits an OOM... and then fires an event that we can listen for, so that we know it happened. The CRITICAL issue is avoiding the OOM EXCEPTION which blows up your app and flow. The whole point of .NET is to make things easy/sensible. And this is one area that REALLY should be handled elegantly by .NET runtime.

I can understand how it might be too difficult for .NET to:

1. Catch an OOM
2. Compact the LOH right there on-the-spot (waiting for all other threads to allow it -- which may not happen if there is a deadlock situation associated with the thread that induced the OOM)
3. And then retry the allocation.

I imagine the only way to make that happen, might be to do a partial-LOH compaction (compact whatever seems safe, to make at least ONE GAP that fits the new object. But that's a lot of new code, for a sloppy result.

So, maybe if instead .NET just resorted to a MM file, to fill-the-gap, and then let the App manually handle the event, so that we can eliminate usage of the Disk-cache very quickly. This seems like a LOT easier solution for the .NET runtime, but would be IMMENSELY USEFUL, to know that you know no longer need to be concerned about OOM's destroying your App's flow.

[msedi]

@najak3d. I agree something needs to be done. My problem is that I'm not aware if everything is currently running as expected and there is indeed a LOH compaction in low memory conditions. As @mangod9 said there already is a LOH compaction. The question is if it is too late in some conditions and you therefore get an OOM. But this is pretty hard to find out.

[najak3d]

Over the years, and even now, it seems that most .NET memory intensive programs have inappropriate/difficult issues with OOMs, and their answer is always to augment .NET's LOH with custom code, or paranoid logic. But if .NET simply handled the OOM in a different manner, most of this concern disappears entirely. All a .NET app should be concerned about is whether or not they are truly "using up all the RAM".

IMO, the runtime GC should have the following API:

1. bool CompactLOHOnce -- already exists. Kudos!
2. bool UseDiskCacheOnOutOfMemory - if true, .NET would use DiskCache instead of throwing an OOM exception.
3. event FullCollection -- fires for every full collection that occurs.
4. event CacheAllocation -- fires each time disk cache is used for an allocation

Event Arguments for the above:

1. FullCollection event:
   TimeSpan TimeConsumed (so you know the cost of the full collection - this way, if your app hiccups, you'll know the cause)
   long TotalBytesAllocated (includes small objects and large objects)
   long TotalBytesAvaliable
   long TotalBytesAvailabeIncludingCache -- this would give you the absolute Max before an OOM exception actually occurs
   (Also - perhaps these values could also be found statically, via GC.LastFullCollectionEvent - and see the same data.

2. CacheAllocation event:
   long CacheBytesAllocated
   long TotalCacheBytesAllocated
   long TotalCacheBytesRemaining

Then, finally, .NET apps will be able to easily manage Memory, without writing excessive/paranoid/tedious code to augment .NET's own memory management logic. As a personal .NET lover-to-the-max, it's always seemed a shame that .NET falters needlessly when it comes to supporting memory-intensive apps.

I am the original author, and co-founder of the Visual3D.NET engine (dead now, killed by Unity3D):

https://vimeo.com/8491336

(that's me narrating; intro was my daughter)

Our demise was exacerbated by .NET's poor support for high-memory-use apps.