Asp.net Core not Collecting Garbage

[zorthgo]

I can't understand why Asp.net core doesn't seem to be collecting garbage. Last week I let a web service run for a few of days, and my memory usage reached 20GB. GC doesn't seem to be working properly. So to test this I wrote a very simple web method that return a large collection of strings. The application started off using only 124MB, but with each time I called the web method, the memory usage kept getting higher and higher until it reached 411MB. It would have gone higher if I had kept calling the web method. But I decided to stop testing.

My test code was this:

`

[HttpGet]
public async Task<IEnumerable> TestGC() {
const string message = "TEST";
return Enumerable.Repeat(message, 100000000);
}
`

Although I might be overlooking something... To my understanding, the memory usage should not be increasing with every call to this method. After the object is created and sent to the UI, the memory should have been freed.

As you can see from the screenshot below, even after the GC got called, the memory was not released.

Thanks for the help!
Jay

[Tratcher]

@rynowak

[davidfowl]

Although I might be overlooking something... To my understanding, the memory usage should not be increasing with every call to this method. After the object is created and sent to the UI, the memory should have been freed.

The large object heap is likely biting you here. If you allocate objects > 85KB in size then it'll be put in the LOH and get compacted very rarely. See http://stackoverflow.com/questions/8951836/why-large-object-heap-and-why-do-we-care for more details (or https://github.com/dotnet/coreclr/blob/master/Documentation/botr/garbage-collection.md#design-of-allocator if you want to go deeper).

I can't understand why Asp.net core doesn't seem to be collecting garbage. Last week I let a web service run for a few of days, and my memory usage reached 20GB

What is your service doing that it's creating such large objects? You should try taking a memory dump before it gets too big, that will clearly show you what objects are sticking around and why it's being held onto (you can use visual studio to look at dumps or a more advanced tool like windbg or perfview).

[MhAllan]

Try to allocate the array from the beginning rather than calling Enumerable.Repeat
or compact the memory using GCSettings.LargeObjectHeapCompactionMode (supported in .Net Standard)

[zorthgo]

Thanks @davidfowl and @MhAllan for the replies. But this example was a contrived one. I just wanted something that would use a noticeable amount of memory so that I could take a screenshot. The truth is that this is happening with any application regardless of the size of the object in question. And to answer your question @davidfowl, my service was just pulling some data from the database with dapper, and returning the resulting object. It was one row of data for each call. So it took a few days for the memory to grow to that amount. I was actually trying to test the DB when I stumbled onto this peculiarity. I had written a little console app at kept calling the method over and over again.

[wanton7]

@zorthgo sure it's not Dapper? If you create your scripts by injecting parameters directly to your SQL scripts like in PHP you'll end up with lot of cached SQL scripts. Here is how Dapper does it
https://github.com/StackExchange/Dapper/blob/fe5c270aceab362c936456087a830e6fe1603cac/Dapper/SqlMapper.cs
You should use a memory profiler to tell what is keeping references to allocated memory. Visual Studio 2017 should be able to help you just take some snapshots from memory before and after multiple calls to your app and compare those.

[Pinox]

@zorthgo Yes I have also seen this. Was using servicestack in my .net core console app and every time i make a call to the api the memory usage increased by massive amounts 50mb. I assumed it was a VS2017 bug but then confirmed the high usage in the task manager. As zorthgo stated by just making simple calls to api the memory usage increase significantly and does not seem to release memory.

[davidfowl]

Does this only happen on ASP.NET Core on .NET Core or is it also A problem with ASP.NET Core on .NET Framework?

Can you write a similar application using MVC 5 (on System.Web) and verify that you don't see the same behavior?

I can't make any progress from this issue in its current state.

[Pinox]

In my instance I i was using only target framework .NetCoreApp 1.1 and my console app was referencing an object model in a shared project.

[Pinox]

Not sure if this will help anyone. a sample app that calls hellorequest. In this app the startup memory is 85mb , then by repetitive request i managed to push up the memory use to about 145mb. It falls back sometimes to 125mb but then stays there. Not sure if this is normal behavior as I'm not used to .Net Core console apps. i always assumed I was doing something wrong or not instantiating correctly.

https://drive.google.com/open?id=0B0Gm-m-z_U84TU5keWFTMzc1ZWc

[codemasta]

Facing same issue here on an Asp.Net Core application deployed to production with 3000-5000 active users..the memory on the server increased to 15GB yesterday...i have had to configure IIS to recylce the AppPool every 3 hours while i still try to figure out what the issue is.

[davidfowl]

Did anybody take a memory dump and look at what is taking up all of the memory in your particular applications?

[ikourfaln]

@davidfowl @Pinox
I'm working on a big company, and we want to start a new project with ASP.NET Core, But when I saw this issue, I was afraid 😟 . this is a critical issue and can block lifecycle of our project.

So please, is it related to ASP.NET Core or .NET Core (CoreCLR) ? we will target Full .NET (4.6) that's why I'm asking.

[Pinox]

@ikourfaln in my case I was using .net core console app , servicestack (.net core version) and kestrel. The weird thing is the memory use goes up to a level , then it stops suddenly and it does not go up again. I guess best is to test it on your side with small sample and check behaviour.

Perhaps @zorthgo can check on his side if he sees similiar behaviour in that memory is used to a certain level and then stops increasing as that is the behaviour that I'm seeing. I have updated my sample app to include @zorthgo example and I dont see the memory running away. It goes up but eventually stops.

I did change the source slightly:

public object Any(TestGC request)
{
const string message = "TEST";
return Enumerable.Repeat(message, 100000);
}

[ikourfaln]

@Pinox
Thank you, I will check behavior on my side.

[CShepartd]

How about this bug in 2.0?