.NET 7: Tiered compilation causes high memory usage and slowdown

[atarz]

Hello, first of all apologies for my bad english.

As a civil engineer and software developer, we can work with very large unknown numbers in building analysis.

I wanted to test the new gc region in net 7 with a simple loop like the one below. It's not real code but I wanted to see how it would work in LOH.

    for (int i = 0; i < 1000; i++)
        {
            double[] bak = new double[10000000];

            for (int j = 0; j < bak.Length; j++)
            {
                var tut = bak[j];
            }
        }

dotnet-sdk-7.0.100-rc.1.22378.2-win-x64
ServerGarbageCollection enable

net 7 is twice as slow as net 6

peak memory on net 6=40-60 mb
peak memory on net 7 =1.2 gb

I follow your work with admiration, thank you very much in advance.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Hello, first of all apologies for my bad english.

As a civil engineer and software developer, we can work with very large unknown numbers in building analysis.

I wanted to test the new gc region in net 7 with a simple loop like the one below. It's not real code but I wanted to see how it would work in LOH.

    for (int i = 0; i < 1000; i++)
        {
            double[] bak = new double[10000000];

            for (int j = 0; j < bak.Length; j++)
            {
                var tut = bak[j];
            }
        }

dotnet-sdk-7.0.100-rc.1.22378.2-win-x64
ServerGarbageCollection enable

net 7 is twice as slow as net 6

peak memory on net 6=40-60 mb
peak memory on net 7 =1.2 gb

I follow your work with admiration, thank you very much in advance.

Author:	atarz
Assignees:	-
Labels:	tenet-performance, area-GC-coreclr, untriaged
Milestone:	-

[mangod9]

we had a recent regression in this with servergc enabled. @mrsharm could you please verify that your latest fix resolves this? Thx!

[mrsharm]

Hi @atarz,

I am unable to repro this issue. Could you please provide the commit hash for the runtime version you used i.e., dotnet-sdk-7.0.100-rc.1.22378.2-win-x64? Also, could you please collect a trace with the issue?

To repro this, I created a console app with the aforementioned code using the bits from the latest main branch of the runtime, enabled server GC and observed the peak working set via Task Manager and it was stagnant on my machine at 25KB.

Thanks in advance!

[atarz]

Hi mrsharm

•Runtime Version: V 7.0.22.40210

The trace I obtained is attached. I don't have much experience, I hope I succeeded.

I tested it on wpf. Today I tested it in the console application with the last main branch. Nothing changed.

In addition to high memory usage, with or without server GC enabled, net 7 completes the process much slower than net 6.
On my computer, net 7 completes the process in 7.5 seconds and net 6 in 4.7 seconds.

I'm getting similar results on different computers.

trace.zip

Thanks for your hard work

[mrsharm]

Hi @atarz,

I am still taking a look at this issue and have made a couple of observations based on taking a look at your trace and my attempt at repro'ing:

1. Since we are specifically allocating on the LOH, we aren't compacting and all the GCs are Background GCs which could explain the high increase in memory.
2. In my repro and the trace provided, I am observing a low GC pause time based on the total duration of the trace implying that the GC :

My repro comparing .NET 6 and .NET 7:

3. I am observing pretty comparable numbers between 6 and 7 in terms of:

4. Allocations

5. Total GC Pause MSec

6. Max Heap Size

And a better time in .NET 7 for the % Time Paused for GC'ing.

4. Taking a look further at the Thread Time Stacks to discern what's consuming the most CPU, I notice that work done on the UI thread that could influence the overall time of the application:

To further help with this, could you please provide the following:

1. A similar trace for the baseline i.e. for your repro for .NET 6.
2. Any information about any other GC configurations you are running for both the .NET 6 and .NET 7 run.
   • As an interesting exercise to see if your memory decreases, you can try setting COMPLUS_GCConserveMemory to 8 or 9 that'll force compaction for the LOH.
3. Any information about your machine such as the memory.

Thanks in advance!

[atarz]

Hi @mrsharm
I think I found the source of the problem. When I disable TieredCompilation everything goes back to normal and I get faster and less memory usage than net 6.

I've tried this on many different machines. 8 cores 4 cores, 16gb 32gb memory, i7 , i5 i9 processor. I got similar results on all of them.

I ran the process in the background and it had no effect

net 7
TieredCompilation:true
ServerGarbageCollection:true
Max peek memory:2.4 gb
Process time:5.3 second

net 7
TieredCompilation:false
ServerGarbageCollection:true
Max peek memory:23 mb
Process time:2.1 second

net 6 TieredCompilation:true or false
ServerGarbageCollection:true
Max peek memory:30 mb
Process time:2.9 second

Thank you very much for your trouble.