-
-
Notifications
You must be signed in to change notification settings - Fork 193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
30 seconds freeze when opening binlog created from VS #739
Comments
I can't reproduce this, when you load the binlog and press F5 to reload, is it fast the next time? We are using Server GC to optimize for loading very large binlogs, but it might create longer GC pauses. Very strange though for such a small binlog. Could it be the antivirus scanning it? Try excluding a directory from scanning and opening the binlog from that directory. |
No, I still have the similar freeze.
No, that didn't help. I added both folder with LogViewer and folder with binlog to Windows Defender exclusions. I don't have other software which could interfere like that. And I also checked Task Manager - there is nothing suspicious there, no other process is taking CPU resource during the freeze. I could also note that in the past (like, couple months ago) I didn't see such problem. Though I don't know what version of LogViewer I used back then - there was no major changes in my laptop setup - only regular driver/OS updates and stuff like that. I did some more profiling and I can see the following issue: While another thread (supposedly the GC thread) is waiting for CPU (waiting to be scheduled by .NET runtime/OS): |
Also, does it happen on a different machine? Does it happen with different binlogs? Can anyone else reproduce this other than you? |
Disabling server GC fixes the issue.
Yes, it happens with all binlogs and repro is stable.
I checked it on colleagues machine - it doesn't reproduce on their machine. We also discovered that setting |
@Maoni0 is this something you’d be interested to look into? |
Small update from me. Still I have no clue why it happens on my machine. I didn't set any system-wide settings. I don't even know which settings could lead to such behaviour. |
It is possible that when starting program from another .NET application it inherits some environment variables similar to |
if the issue is fixed by setting noaffinitize to true, it usually indicates something else is preventing the GC threads from running. could you capture a ThreadTime trace with perfview that covers that period of time (doesn't have to be the whole 30s, but some of it would do) perfview /ThreadTime /BufferSizeMB:4000 /CircularMB:4000 /nogui collect |
Hi. Sure, please see the attached snapshot: (I had to disable Windows Defender's real-time protection though, otherwise I was getting "The requested resource is in use" ComException when starting perfview) |
thanks for the trace. so both GCs are induced. you can see this if you open the trace in perfview and look at the GCStats view
and if you open the Any Stacks view and look at the GC/Triggered stacks you'll see they are triggered by so you might want to capture a trace to see if you see these induced GCs in the other scenarios. and the reason why they take so long is there's a very high priority thread (priority 15) from System that runs some nvidia stuff (no symbols so we can't tell what it's actually doing but it's from nvlddmkm. I explained how to debug this kind of issues here: https://github.com/Maoni0/mem-doc/blob/master/doc/.NETMemoryPerformanceAnalysis.md#debugging-a-random-long-gc. if you are currently using HW acceleration for gfx, you could try to switch to SW see if that helps. but otherwise that is the problem. |
Thanks for looking into it! I tried to use both NVIDIA and Intel GPU (for this process) - the issue is reproducible in both cases. I sent a bug report to NVIDIA so hopefully they will take a look at it. They don't have any public bug tracker so I cannot share a link or ticket number here. I also wonder, if the application enables GC server mode but application is intended to be used on a client machine (with more varied and less controlled environment), whether it makes sense to also set |
that's a valid argument. you could even say that if you specified the GCHeapCount config, GC should just make the default of the GCNoAffinitize config true instead of false. I would say this is really just due to historical reasons - Server GC is already used way less on client machines so we basically just depend on folks who do use it to do their own configuration. for example, VS now uses Server GC and specifies both the GCHeapCount and the GCNoAffinitize config (to true). btw, I see you have 16 heaps so I'm guessing the GCHeapCount config is actually not specified. it doesn't seem like an app like this should need so many GC heaps. btw, in a memory constraint scenario, eg, an app running in a container with a memory limit, GCNoAffinitize is by default true. I could change this default when the GCHeapCount config is specified as mentioned above, of course that would make it sort of a breaking change (if you believe perf changes can be categorized as breaking) for folks who did rely on it being false. |
The reason why I had GC.Collect() there in the first place is that sometimes we open huge binlogs, which make the commit memory grow to 50 GB (so you must have at least 64 GB RAM to open those binlogs on that machine). After a binlog is read, we no longer need a bunch of data structures, but the GC didn't happen immediately, so I was letting it know I'm done allocating and we can free up all that unnecessary memory after loading the binlog. We also found large binlogs open almost twice as fast with Server GC (19 seconds vs. 30 seconds), that's why we turned Server GC on. |
Yes, that parameter is not specified.
Is it possible to experiment how changing |
Hi, I'm inclined to close this as it's not really a bug in the viewer. Your external machine state was slowing down GC that normally happens fast. |
Binlog is created on empty .NET solution via Project System Tools 2022 extension (Version: 1.0.2.2305901).
Binlog size is 60 KB.
In the snapshot I see that 30 seconds freeze is happening due to GC which is really weird since there are just 40 MBs of allocations.
Program version: 2.2.155
I'm attaching repro solution, binlog, snapshot (from dotTrace) (on Google Drive) and its screenshot:
The text was updated successfully, but these errors were encountered: