-
Notifications
You must be signed in to change notification settings - Fork 355
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dotnet-dump "mini" dumps do not include enough data to debug managed process dumps #3065
Comments
FULL dump:
MINI dump:
|
If the singlefile app doesn't have a DAC next to it, it needs to get extracted from the host - we'd need make sure it's there, it's getting extracted, and we'd need to get a BP here https://github.com/dotnet/runtime/blob/9d972eda4da654dfafae101e9850b1191a4dcbf9/src/coreclr/debug/daccess/enummem.cpp#L1535. This needs a recent enough version of dbghelp to work. |
Yes, this feels like the DAC isn't be loaded when the dump is generated (it shouldn't matter whether dotnet-dump/createdump or windbg is used to generate it). |
This was generated with:
It's possible the dac wasn't loaded, but it feels strange that this is the case. The dac is embedded in the single_file.exe, I did double check that. I would have imagined that dotnet-dump (or the part of CLR that does the dumping) would do the right thing here. We definitely need to dig deeper to see what's going on, something is busted with the "out of the box" single file minidump scenario, I'm just not sure where the issue is. |
dbgcore comes from the machine and needs to have support for that embedded DAC. We don't redistribute it, and it does lead to pits of failure like this one - it could be interesting debating redistributing this, as long as we have a mechanism to service it in a timely manner as needed. We'd need to ask Tim and folks what their thoughts are. |
At Juan's suggestion I added dbgeng.dll and dbgcore.dll next to dotnet-dump which changed the behavior but it still fails. .dumpdebug shows that we put a little extra bytes into the dump:
Full reads are below, but at its core we fail to read memory in this callstack:
Not enough of the PE header was written into the dump, and presumably not the exports or the resource table. I haven't put dotnet-dump under a debugger yet, but I have to leave for the day.
|
So I think there may be two issues here:
I will leave 1) to you all. I will dig further into 2) above tomorrow and see exactly what we put into the dump when those dlls exist and if it's the right data I'll see if we can re-write ClrMD to parse it successfully. Also please note that in the original bug the user reported that "WinDbg and SOS work fine", this is likely due to running it on the same PC as the dump and WinDbg smartly memory maps the local image from disk. ClrMD should do that too (and we'll fix it) but there's still a bug here. Moving the dump to a different PC (or deleting that .exe from disk that WinDbg is memory mapping) would reproduce the problem too I think. |
WinDBG does know how to get a very thin slice of the export for this purpose. |
Juan, doesn't dbghelp.dll need to be side-by-side with the dotnet-dump exe? It does a pinvoke to MiniDumpWriteDump in dbghelp.dll. dbgeng/dbgcore are also needed too. Lee, I have also had similar problems with using the System.Reflection.Metadata. SOS only uses it on an actual module file. Otherwise, SOS uses the symstore PEFile reader which doesn't read the whole header and all the data directories up front. It only reads exactly what is necessary. It is very lazy. |
Ok I've eliminated 2) as an issue. Replacing System.Reflection.Metadata with my own PEImage reader resulted in correctly loading the image. However, once I got further it's clear that not enough data is put into the dump, presumably the issue with 1) is that the dac wasn't loaded/called. Happy to help further here if there's other issues suspected with ClrMD, but I go back to the original issue: Not enough data is put into the dump when you run Whether that's simply because dbg*.dll isn't present or other issues I leave up to you all. =) |
The change to the title of this bug does not reflect reality, I've changed it back. Dotnet-dump creates crash dumps. If those crash dumps don't contain enough data to debug the underlying crash, then this is a bug. Dotnet-dump isn't doing what it clams to do (generate debuggable crash dumps). It's possible that you could solve this by shipping dbgeng/dbgcore along with dotnet-dump. That's a possible solution, but it's not the issue that I'm reporting. |
It is against Windows policy to ship/redistribute the fixed dbghelp/dbgcore/dbgeng with dotnet-dump. This fix will shipping in a future Windows service release. |
Description
In order to properly debug a crash dump generated by single-file exes, we need more data from the executable's PE image. Specifically we need to read the export table to know if any given module is a single file module, e.g.:
https://github.com/leculver/clrmd/blob/25dea09efb3756ec46f4676fee340067ab85b228/src/Microsoft.Diagnostics.Runtime/src/DataTargets/DataTarget.cs#L190
When the PE image data is not present in the dump we will not be able to know whether a particular image is single file or not.
dotnet-dump
will need to detect the single file case and add enough of the PEImage header so that we can walk the export table and get the runtimeinfo.Additionally the section of memory for the RuntimeInfo needs to be in the dump as well.
Repro
Build this project: https://github.com/microsoft/clrmd/tree/master/src/TestTargets/single-file
With this command line:
dotnet publish -c Release -o publish -p:PublishReadyToRun=true -p:PublishSingleFile=true -p:PublishTrimmed=true --self-contained true -p:IncludeNativeLibrariesForSelfExtract=true
Then use
dotnet-dump --type Mini
to create a crashdump.Then load the crash dump with this code:
DataTarget.LoadDump(@"C:\path\to\crash.dmp").ClrVersions[0].CreateRuntime();
.This will crash with an index out of range exception, with ClrVersions.Length == 0, which is due to not being able to create a Microsoft.Diagnostics.Runtime.PEImage due to no data being in the dump for that image. Enough data needs to be present in the dump such that we succeed to read the export table.
The text was updated successfully, but these errors were encountered: