How to locate coreclr.dll from hostfxr_initialize_for_runtime_config

[rseanhall]

TL;DR What's the correct way to locate coreclr.dll using the results of a successful call to hostfxr_initialize_for_runtime_config?

I am writing a custom .NET Core host for wixtoolset/issues#6108. The use case here is that the main application (the Burn engine, or burn.exe) is a native application that loads basically another application (the Bootstrapper Application, or BA) from a DLL. Both Burn and the BA run on their own threads and communicate with each other to get the job done. So my custom host needs to load the .NET Core runtime, instantiate the managed code BA (from a user provided DLL), and return essentially a handle to that managed BA to the native engine. This was implemented long ago for the full .NET Framework here.

Since I need to load the managed code as a component, I started with hostfxr_initialize_for_runtime_config with an self-contained deployment (SCD) style test project. I quickly learned, however, that hostfxr_initialize_for_runtime_config doesn't support a .runtimeconfig.json for SCD. (Why does my custom host need to know whether the target was deployed by the SCD or FDD (framework-dependent deployment) style?) I eventually got SCD working by not using the nethost/hostfxr/hostpolicy layer.

So I came back to hostfxr_initialize_for_runtime_config for testing FDD style projects, since I need the logic of finding the appropriate framework based on the requirements of the target managed application. The initialization worked great, but I had trouble with load_assembly_and_get_function_pointer. (Why do the type and delegate parameters require an assembly qualification, when I'm required to provide the assembly path as a parameter to the same function?)

After working that out, I successfully got to the managed side of this custom host. But I ran into some issues trying to load the target managed DLL. The original code was using AppDomain.Current.Load, which returned E_FILE_NOT_FOUND. (AppDomain.Current.Load was working perfectly when I loaded the framework directly through coreclr for the SCD workflow, why would it behave differently through hostfxr_initialize_for_runtime_config?) Then I switched to Assembly.Load and eventually started setting the APP_PATHS runtime property with the target directory. I don't remember which combinations produced which errors, but sometimes a dependent assembly wasn't getting loaded and sometimes that dependent assembly was getting loaded twice. (Are AppDomain.Current.Load and Assembly.Load supposed to work differently? )

Because of those managed assembly load issues with hostfxr_initialize_for_runtime_config, and plus the SCD workflow can't use it either, I want to use coreclr directly in both scenarios. But I still need hostfxr_initialize_for_runtime_config in the FDD workflow so that it can tell me which coreclr.dll to load. What is the correct way to do that? I have called hostfxr_get_runtime_properties and there are multiple properties that I could use to guess where coreclr.dll is. For instance, go through the TPAs and get the parent directory of a certain DLL (but which DLL?). Or get the parent directory of JIT_PATH (is this always going to be available in future versions?). Or use the directories in NATIVE_DLL_SEARCH_DIRECTORIES (it includes C:\ which seems weird to me).

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. Please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @vitek-karas, @swaroop-sridhar
Notify danmosemsft if you want to be subscribed.

[AaronRobinsonMSFT]

/cc @elinor-fung

[rseanhall]

I've been researching my managed assembly loading issues with hostfxr_initialize_for_runtime_config/hostfxr_get_runtime_delegate/load_assembly_and_get_function_pointer. I now understand they were due to my complete ignorance around the AssemblyLoadContext concept, as well as the undocumented functionality of hostfxr_get_runtime_delegate/load_assembly_and_get_function_pointer loading the assembly and the delegate into an isolated AssemblyLoadContext.

That doesn't help me with my core issue, though. I need a supported way to call into my managed entry point, such that the managed environment is the same regardless of whether the app was published FDD or SCD. This custom host could easily end up in an installer built this year that someone tries to use 10 years now with .NET Core 2030. I can think of 3 options:

1. Ideally hostfxr_initialize_for_runtime_config would have supported SCD. It's hard for me to propose an alternative API that supports both FDD and SCD here since I haven't found any explanation for this limitation. Perhaps hostfxr_initialize_for_scd_directory? If this new API can make it into .NET 5, this would be the preferred option.

2. An algorithm for locating coreclr.dll from hostfxr_initialize_for_runtime_config is put in documentation somewhere so that it will continue to be supported by future versions of the framework.

3. The current implementation of hostfxr_get_runtime_delegate/load_assembly_and_get_function_pointer gets publicly documented so that it will continue to be supported by future versions of the framework. By this I mean I could have my SCD implementation call coreclr_create_delegate for the LoadAssemblyAndGetFunctionPointer method of Internal.Runtime.InteropServices.ComponentActivator in System.Private.CoreLib.

[vitek-karas]

Is this problem solvable via the proposed solution mentioned in #35465? (I assume these two issue represent the same scenario - right?)

Just for completeness to answer your questions:

1. Currently .NET Core SDK doesn't support building SCD component projects (classlib and similar) - so having a support for it in the host/runtime feels wrong as there is no good way to create one.
2. Right now it's intentional that we don't expose a standard way to determine the location of coreclr.dll from the hostfxr APIs. We think that using the combination of hostfxr and direct calls to coreclr would lead to lot of confusion. Ideally people should use just the hostfxr APIs. I do agree that there's a small area where those APIs are not good enough and where direct interaction with coreclr is a viable alternative - one way is part of the proposal here: Log details of app being launched by dotnet(.exe) to ETW/LTTng #3156
3. I think this is covered by the discussion in BinaryFormatter Deserialize Datatable throw "Type * is not deserializable." Exception  #35346 .

[rseanhall]

Yes, I created both of these from the same scenario so I shouldn't need this anymore. However, someone else might have a scenario where they need to use hostfxr APIs to select the framework but can't use hostfxr to start the runtime (maybe they need to customize the startup flags or something). I don't understand how the ETW issue is related to this - it's asking for the host to send information to the runtime but the runtime will already know where it is.

[vitek-karas]

Customize startup flags: This is already fully supported by the hostfxr APIs - the context returned by the hostfxr_intialize_* apis can be used to read and write the runtime properties passed to the coreclr_intiialize - see hostfxr_get_runtime_property_value and hostfxr_set_runtime_property_value.

Issue #3156 will add a new runtime property (I think the last proposed name was something like RESOLVED_FRAMEWORKS) which can be accessed via the hostfxr_get_runtime_property_value and it will contain the path to the Microsoft.NETCore.App - that framework will contain the coreclr.dll in it (which is VERY unlikely to ever change).

[rseanhall]

Looking at #644, I don't see any path information in RESOLVED_FRAMEWORKS. There would be no Microsoft.NETCore.App in SCD anyway. FWIW, I'm going to be using NATIVE_DLL_SEARCH_DIRECTORIES in 3.x.

[rseanhall]

This doesn't seem to be going anywhere. Like I said, I don't believe #3156 is going to help here because it's not going to be adding anything with path information.

This can easily be accomplished by getting hostfxr to create a CORECLR_PATH runtime property with the full path to coreclr.dll. If I sent a pull request for that, would you accept it?

[vitek-karas]

Maybe a better issue to reference would be #33472. The underlying implementation is similar as both needs to expose information about frameworks in the runtime properties. This one explicitly mentions paths to be included.