hostfxr.dll loading failing on Windows 11 Arm64 with .NET 6 RC2

[richlander]

hostfxr.dll loading failing on Windows 11

Using https://github.com/dotnet/core/blob/main/samples/dotnet-runtimeinfo.

This is what I see.

C:\git\core\samples\dotnet-runtimeinfo>\dotnet5arm64\dotnet run
**.NET information
Version: 5.0.9
FrameworkDescription: .NET 5.0.9
Libraries version: 5.0.9
Libraries hash: 208e377a5329ad6eb1db5e5fb9d4590fa50beadd

**Environment information
OSDescription: Microsoft Windows 10.0.22000
OSVersion: Microsoft Windows NT 10.0.22000.0
OSArchitecture: Arm64
ProcessorCount: 8


C:\git\core\samples\dotnet-runtimeinfo>\dotnetarm64\dotnet run
Failed to load the dll from [C:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll], HRESULT: 0x800700C1
The library hostfxr.dll was found, but loading it from C:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll failed
  - Installing .NET Core prerequisites might help resolve this problem.
     https://go.microsoft.com/fwlink/?linkid=798306

C:\git\core\samples\dotnet-runtimeinfo>dir C:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4
 Volume in drive C is Windows
 Volume Serial Number is F64D-24D0

 Directory of C:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4

08/20/2021  09:35 AM    <DIR>          .
08/20/2021  09:35 AM    <DIR>          ..
08/20/2021  09:21 AM           380,928 hostfxr.dll
               1 File(s)        380,928 bytes
               2 Dir(s)  90,463,694,848 bytes free

C:\git\core\samples\dotnet-runtimeinfo>\dotnetarm64\dotnet bin\Debug\netcoreapp3.1\dotnet-runtimeinfo.dll
**.NET information
Version: 6.0.0
FrameworkDescription: .NET 6.0.0-rc.2.21420.4
Libraries version: 6.0.0-rc.2.21420.4
Libraries hash: aa219d8aa2a03726e02a643cf8b36fec13d2d484

**Environment information
OSDescription: Microsoft Windows 10.0.22000
OSVersion: Microsoft Windows NT 10.0.22000.0
OSArchitecture: Arm64
ProcessorCount: 8

C:\git\core\samples\dotnet-runtimeinfo>set COREHOST_TRACE=1

C:\git\core\samples\dotnet-runtimeinfo>set COREHOST_TRACEFILE=host.txt

C:\git\core\samples\dotnet-runtimeinfo>set DOTNET_ROOT=c:\dotnetarm64

C:\git\core\samples\dotnet-runtimeinfo>bin\Debug\netcoreapp3.1\dotnet-runtimeinfo.exe
Failed to load the dll from [c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll], HRESULT: 0x800700C1
The library hostfxr.dll was found, but loading it from c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll failed
  - Installing .NET Core prerequisites might help resolve this problem.
     https://go.microsoft.com/fwlink/?linkid=798306

C:\git\core\samples\dotnet-runtimeinfo>type host.txt
Tracing enabled @ Fri Aug 20 23:51:02 2021 GMT
--- Invoked apphost [version: 3.1.18, commit hash: 5d3919d34e0d8940ee50c36d823ddc28d6b1723c] main = {
bin\Debug\netcoreapp3.1\dotnet-runtimeinfo.exe
}
Redirecting errors to custom writer.
The managed DLL bound to this executable is: 'dotnet-runtimeinfo.dll'
Using environment variable DOTNET_ROOT=[c:\dotnetarm64] as runtime location.
Reading fx resolver directory=[c:\dotnetarm64\host\fxr]
Considering fxr version=[6.0.0-rc.2.21420.4]...
Detected latest fxr version=[c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4]...
Resolved fxr [c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll]...
Failed to load the dll from [c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll], HRESULT: 0x800700C1
The library hostfxr.dll was found, but loading it from c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll failed
  - Installing .NET Core prerequisites might help resolve this problem.
     https://go.microsoft.com/fwlink/?linkid=798306

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

hostfxr.dll loading failing on Windows 11

Using https://github.com/dotnet/core/blob/main/samples/dotnet-runtimeinfo.

This is what I see.

C:\git\core\samples\dotnet-runtimeinfo>\dotnet5arm64\dotnet run
**.NET information
Version: 5.0.9
FrameworkDescription: .NET 5.0.9
Libraries version: 5.0.9
Libraries hash: 208e377a5329ad6eb1db5e5fb9d4590fa50beadd

**Environment information
OSDescription: Microsoft Windows 10.0.22000
OSVersion: Microsoft Windows NT 10.0.22000.0
OSArchitecture: Arm64
ProcessorCount: 8


C:\git\core\samples\dotnet-runtimeinfo>\dotnetarm64\dotnet run
Failed to load the dll from [C:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll], HRESULT: 0x800700C1
The library hostfxr.dll was found, but loading it from C:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll failed
  - Installing .NET Core prerequisites might help resolve this problem.
     https://go.microsoft.com/fwlink/?linkid=798306

C:\git\core\samples\dotnet-runtimeinfo>dir C:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4
 Volume in drive C is Windows
 Volume Serial Number is F64D-24D0

 Directory of C:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4

08/20/2021  09:35 AM    <DIR>          .
08/20/2021  09:35 AM    <DIR>          ..
08/20/2021  09:21 AM           380,928 hostfxr.dll
               1 File(s)        380,928 bytes
               2 Dir(s)  90,463,694,848 bytes free

C:\git\core\samples\dotnet-runtimeinfo>\dotnetarm64\dotnet bin\Debug\netcoreapp3.1\dotnet-runtimeinfo.dll
**.NET information
Version: 6.0.0
FrameworkDescription: .NET 6.0.0-rc.2.21420.4
Libraries version: 6.0.0-rc.2.21420.4
Libraries hash: aa219d8aa2a03726e02a643cf8b36fec13d2d484

**Environment information
OSDescription: Microsoft Windows 10.0.22000
OSVersion: Microsoft Windows NT 10.0.22000.0
OSArchitecture: Arm64
ProcessorCount: 8

C:\git\core\samples\dotnet-runtimeinfo>set COREHOST_TRACE=1

C:\git\core\samples\dotnet-runtimeinfo>set COREHOST_TRACEFILE=host.txt

C:\git\core\samples\dotnet-runtimeinfo>set DOTNET_ROOT=c:\dotnetarm64

C:\git\core\samples\dotnet-runtimeinfo>bin\Debug\netcoreapp3.1\dotnet-runtimeinfo.exe
Failed to load the dll from [c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll], HRESULT: 0x800700C1
The library hostfxr.dll was found, but loading it from c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll failed
  - Installing .NET Core prerequisites might help resolve this problem.
     https://go.microsoft.com/fwlink/?linkid=798306

C:\git\core\samples\dotnet-runtimeinfo>type host.txt
Tracing enabled @ Fri Aug 20 23:51:02 2021 GMT
--- Invoked apphost [version: 3.1.18, commit hash: 5d3919d34e0d8940ee50c36d823ddc28d6b1723c] main = {
bin\Debug\netcoreapp3.1\dotnet-runtimeinfo.exe
}
Redirecting errors to custom writer.
The managed DLL bound to this executable is: 'dotnet-runtimeinfo.dll'
Using environment variable DOTNET_ROOT=[c:\dotnetarm64] as runtime location.
Reading fx resolver directory=[c:\dotnetarm64\host\fxr]
Considering fxr version=[6.0.0-rc.2.21420.4]...
Detected latest fxr version=[c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4]...
Resolved fxr [c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll]...
Failed to load the dll from [c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll], HRESULT: 0x800700C1
The library hostfxr.dll was found, but loading it from c:\dotnetarm64\host\fxr\6.0.0-rc.2.21420.4\hostfxr.dll failed
  - Installing .NET Core prerequisites might help resolve this problem.
     https://go.microsoft.com/fwlink/?linkid=798306

Author:	richlander
Assignees:	-
Labels:	area-Host, untriaged
Milestone:	-

[mateoatr]

This looks like an SDK problem. When running dotnet5arm64\dotnet run the SDK downloads the following apphost package:

AppHostPack=
    AppHost
        NuGetPackageId=Microsoft.NETCore.App.Host.win-arm64
        NuGetPackageVersion=3.1.12
        PathInPackage=runtimes\win-arm64\native\apphost.exe
        RuntimeIdentifier=win-arm64 (TaskId:22)

and uses it as the source path for the apphost:

ResolvedAppHostPack=
    AppHost
        NuGetPackageId=Microsoft.NETCore.App.Host.win-arm64
        NuGetPackageVersion=3.1.12
        OriginalItemSpec=AppHost
        PackageDirectory=D:\dotnet\.nuget\packages\microsoft.netcore.app.host.win-arm64\3.1.12
        PathInPackage=runtimes\win-arm64\native\apphost.exe
                                     RuntimeIdentifier=win-arm64 (TaskId:58)

Set Property: AppHostSourcePath=D:\dotnet\.nuget\packages\microsoft.netcore.app.host.win-arm64\3.1.12\runtimes\win-arm64\native\apphost.exe

After that, everything works as expected. The app runs and we get the expected output.

Now, when running dotnet6arm64\dotnet run, the SDK will change the apphost rid from win10-arm64 to win10-x64 (see here).

This gets you a x64 apphost, since we are targeting netcoreapp3.1. Because dotnet run currently sets DOTNET_ROOT to the muxer path, you'll be now trying to run a x64 binary with arm64 bits, hence the error.

About why dotnet <app>.dll works and not executing the exe directly -- my guess is that the produced .dll is x86 and the exe is x64 and we are running in a machine that does not support x64 emulation yet. Could you try opening a developer cmd and do dumpbin /headers dotnet-runtimeinfo.dll | findstr "machine" and dumpbin /headers dotnet-runtimeinfo.exe | findstr "machine" after running dotnet6arm64\dotnet run?

[richlander]

Did some further validation on this app.

.NET 5 provides an Arm64 exe in this case, which makes sense since the apphost and the SDK RID will always match (for .NET 5 behavior).

.NET 6 provides an x64 exe, which also makes sense since we changed the behavior for per-.NET 6 versions (per my own spec).

This works: dotnet run -a arm64. It coerces .NET 6 to produce an Arm64 binary.

Back to the error message. It is very misleading. I thought maybe that the RollForward property was confusing things. It doesn't appear to be. I'd expect an error messaging about the version not matching but that doesn't happen.

Do we need a compatible (x64) hostfxr in order for the apphost to be able to provide a decent error message for an x64 app on an Arm64 system? I fear that this is true.

[vitek-karas]

I looked into this some more - on Windows this should be relatively easy - LoadLibrary returns last error ERROR_BAD_EXE_FORMAT (0x800700C1) which is basically only going to happen on architecture mismatch.

On Linux/Mac this is more difficult. Unlike on Windows the existing error message is more user friendly - on mac we basically print out "mach-o, but wrong architecture", which hints at what's going on. Unfortunately I was not able to find a way to detect this programmatically (the dlerror API only returns a string, and I can't find a way to get something like an error code). We could assume that this is going to be the most common error and simply write the error that way.

In either case, we will only be able to tell that there was an architecture mismatch (if even that), but not what the architecture of the found hostfxr actually is.

So let me turn the tables on you - what would you like to see as an error in this case? And let's see how close we can actually get.

(Technically it's possible to write code which would open the hostfxr file as binary and try to parse the OS headers to tell which architecture it is - on Windows this is pretty simple, I assume it's going to be simple on the other platforms as well. But it would yet more very platform specific code to maintain).

[richlander]

Doesn't all of this boil down to dotnet run not calling the apphost?

I'm now doing this with a global install.

rich@MacBook-Air-M1-2020 dotnet-runtimeinfo % dotnet run
Failed to load /usr/local/share/dotnet/host/fxr/6.0.0-rc.2.21423.6/libhostfxr.dylib, error: dlopen(/usr/local/share/dotnet/host/fxr/6.0.0-rc.2.21423.6/libhostfxr.dylib, 0x0001): tried: '/usr/local/share/dotnet/host/fxr/6.0.0-rc.2.21423.6/libhostfxr.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/local/lib/libhostfxr.dylib' (no such file), '/usr/lib/libhostfxr.dylib' (no such file)
The library libhostfxr.dylib was found, but loading it from /usr/local/share/dotnet/host/fxr/6.0.0-rc.2.21423.6/libhostfxr.dylib failed
  - Installing .NET prerequisites might help resolve this problem.
     https://go.microsoft.com/fwlink/?linkid=2063366
rich@MacBook-Air-M1-2020 dotnet-runtimeinfo % ls bin/Debug 
net5.0
rich@MacBook-Air-M1-2020 dotnet-runtimeinfo % ./bin/Debug/net5.0/dotnet-runtimeinfo
**.NET information
Version: 6.0.0
FrameworkDescription: .NET 6.0.0-rc.2.21423.14
Libraries version: 6.0.0-rc.2.21423.14
Libraries hash: b7df9ff170c2276899456460cf494e2c3ba41940

**Environment information
OSDescription: Darwin 21.0.0 Darwin Kernel Version 21.0.0: Mon Aug  2 00:18:13 PDT 2021; root:xnu-8019.30.9~9/RELEASE_ARM64_T8101
OSVersion: Unix 11.0.0
OSArchitecture: X64
ProcessorCount: 8

rich@MacBook-Air-M1-2020 dotnet-runtimeinfo % file bin/Debug/net5.0/dotnet-runtimeinfo
bin/Debug/net5.0/dotnet-runtimeinfo: Mach-O 64-bit executable x86_64
rich@MacBook-Air-M1-2020 dotnet-runtimeinfo % cat /etc/dotnet/install_location
/usr/local/share/dotnet/x64
arm64=/usr/local/share/dotnet
x64=/usr/local/share/dotnet/x64

I guess these are two separate problems. Once we fix this, then we still have the other problem that you are getting at.

Imagine the Arm64 installer additionally installed an x64 hostfxr and the apphost knew about it. Wouldn't that solve the problem? On second thought, I guess building that (on our part) would be a bit of a nightmare, but it is still worth discussing.

[agocke]

Fixed by dotnet/sdk#19860