PInvoke issue with SHGetFileInfo w/ SHGFI_ICON

[Androvald]

Description

In .NET 9 and 10 calling SHGetFileInfo with the request for an icon crashes the application with exit code -1073740791 (0xc0000409) - stack overflow. This does not happen in other versions of .NET (Tested 6 and 8 specifically).

I found this after debugging the new Powertoys Command palette, since it was crashing whenever I entered anything in the search box.

Reproduction Steps

using System.Runtime.InteropServices;

class Program
{
    public static void Main()
    {

        const uint SHGFI_ICON = 0x000000100;
        const uint SHGFI_DISPLAYNAME = 0x000000200;

        var fileInfo = new SHFILEINFO();
        Console.WriteLine("Starting");
        Console.WriteLine(Marshal.SizeOf(fileInfo));
        var result = SHGetFileInfo(@"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Microsoft Edge.lnk", 0, ref fileInfo, (uint)Marshal.SizeOf(fileInfo), SHGFI_ICON);
        Console.WriteLine(".NET 9 does not reach here.");
        DestroyIcon(fileInfo.hIcon);
        Console.WriteLine(result);
        Console.WriteLine(fileInfo.szDisplayName);
    }

    [DllImport("shell32.dll", CharSet = CharSet.Unicode)]
    static extern nint SHGetFileInfo(string pszPath, uint dwFileAttributes, ref SHFILEINFO psfi, uint cbFileInfo, uint uFlags);

    [DllImport("user32.dll", CharSet = CharSet.Unicode)]
    static extern bool DestroyIcon(IntPtr hIcon);

    [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
    struct SHFILEINFO
    {
        public IntPtr hIcon;
        public int iIcon;
        public uint dwAttributes;
        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 260)]
        public string szDisplayName;
        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 80)]
        public string szTypeName;
    }
}

Expected behavior

Return a handle to an icon in the struct.

Actual behavior

Looks like a stack overflow, since it aborts the process, and does not allow the debugger to do anything.

Regression?

Works in .NET 8 and prior.

Known Workarounds

None

Configuration

Windows 11, 24H2 build 26100.4061 on Intel Core i7-13700K in x64 mode.
Tested .NET 9.0.5 w/ sdk 9.0.300 and .NET 10 preview 4.
Not sure if it's specific to this configuration.

Other information

For some reason this does not crash in Linqpad 8, even in .NET 9 mode. I'm not sure what is different there.
Other options do seem to work fine. Such as the display name comes through just fine if requested. But it breaks the moment you request an icon.

[AaronRobinsonMSFT]

@Androvald I took the example above and added the Console.WriteLine($".NET Runtime Version: {RuntimeInformation.FrameworkDescription}"); statement to the top of Main. I don't see a failure here. I've quickly tried on .NET 10 main and also didn't see an issue. Can you confirm the repro?

...\runtime116151>dotnet run -c Release
...\runtime116151\Program.cs(9,20): warning CS0219: The variable 'SHGFI_DISPLAYNAME' is assigned but its value is never used
.NET Runtime Version: .NET 9.0.5
Starting
696
.NET 9 does not reach here.
1


...\runtime116151>dotnet run -c Debug
...\runtime116151\Program.cs(9,20): warning CS0219: The variable 'SHGFI_DISPLAYNAME' is assigned but its value is never used
.NET Runtime Version: .NET 9.0.5
Starting
696
.NET 9 does not reach here.
1

[Androvald]

Yeah, still happening:

[Androvald]

I would not be surprised if there is something weird in my configuration somehow.

The fact that it only happens in .NET 9 and 10, and not in C++ or .NET 8 and below, and the fact that it only happens when asking for the icon are both very odd though. I did just find that it created crash dumps in LocalAppData, so let me upload one.

TestApp.exe.51752.dmp

Edit: This one is made without a debugger attached, so at least seems to have a managed callstack:
TestApp.exe.576.dmp

[Androvald]

Oh a hunch I tried running it in Visual Studio with the native debugger enabled, and got a far more detailed call stack.
I hope this helps:

TestApp.mini.dmp

[Androvald]

Alright, I figured out the missing piece of the puzzle. I have the PlasticSCM / Unity Version Control client installed, and it adds icon overlays to show the source control status. After removing the registry entries for them the problem goes away.

The shell extension they register is written in .NET Framework 3.5 (according to dotPeek), so I'm guessing there is something that breaks when both .NET 9 / 10 and .NET Framework 3.5 are used in the same process space.

Unfortunately due to the indirect nature of shell extensions, for regular users this incompatibility could be quite bad.

[AaronRobinsonMSFT]

@Androvald Thanks for the DMPs. It isn't immediately obvious the issue here and debugging a DMP with both .NET Core and .NET Framework isn't a supported scenario. In fact having both runtimes in the same process isn't really a supported scenario due to the myriad of diagnostics issues. To add onto all of this, sorry, shell extensions written in .NET aren't supported.

See https://learn.microsoft.com/windows/win32/shell/shell-and-managed-code

One runtime of particular note is the common language runtime (CLR), also known as managed code or the .NET Framework. Microsoft recommends against writing managed in-process extensions to Windows Explorer or Windows Internet Explorer and does not consider them a supported scenario.

Given all the above there isn't much we can do about this. I'm going to close this issue.