<probing privatePath="..." /> doesn't work in .Net 5.0

[rlktradewright]

In .Net 4, probing during assembly loading could be influenced by use of the 'probing' element in the app.exe.config file. The privatePath attribute contained list of subfolders that would be searched during probing. An example from one of my apps is:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2" />
  </startup>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <probing privatePath="TradeWright.TradeBuild.ComInterop" />
    </assemblyBinding>
  </runtime>
</configuration>

In this example, the TradeWright.TradeBuild.ComInterop subfolder contains a number (69 to be precise) of COM interop libraries produced by use of tlbimp.exe and AxImp.exe. The corresponding ActiveX .dlls and .ocxs are in parallel subfolders with appropriate manifests to enable registration-free COM, and the .exes have corresponding embedded application manifests. This all works perfectly, and it means my top-level bin folder contains only the .exes and .exe.configs, and all the interop dll's are neatly tucked away.

However this does not work with .Net 5. Including an app.config file in the project results in an app.dll.config file being generated, but the 'probing' element is not actioned. The interop files are not located and the apps get nowhere.

If I move the interop dllls into the same folder as the .exe, all works fine, but I really don't want the top-level bin folder polluted with all these interop dlls.

Is there any current workaround for this? And is there any plan to provide the same probing behaviour as in .Net 4?

Tagging subscribers to this area: @vitek-karas, @agocke, @CoffeeFlux
See info in area-owners.md if you want to be subscribed.

Issue Details

---

In .Net 4, probing during assembly loading could be influenced by use of the 'probing' element in the app.exe.config file. The privatePath attribute contained list of subfolders that would be searched during probing. An example from one of my apps is:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.7.2" />
  </startup>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <probing privatePath="TradeWright.TradeBuild.ComInterop" />
    </assemblyBinding>
  </runtime>
</configuration>

In this example, the TradeWright.TradeBuild.ComInterop subfolder contains a number (69 to be precise) of COM interop libraries produced by use of tlbimp.exe and AxImp.exe. The corresponding ActiveX .dlls and .ocxs are in parallel subfolders with appropriate manifests to enable registration-free COM, and the .exes have corresponding embedded application manifests. This all works perfectly, and it means my top-level bin folder contains only the .exes and .exe.configs, and all the interop dll's are neatly tucked away.

However this does not work with .Net 5. Including an app.config file in the project results in an app.dll.config file being generated, but the 'probing' element is not actioned. The interop files are not located and the apps get nowhere.

If I move the interop dllls into the same folder as the .exe, all works fine, but I really don't want the top-level bin folder polluted with all these interop dlls.

Is there any current workaround for this? And is there any plan to provide the same probing behaviour as in .Net 4?

Author:	rlktradewright
Assignees:	-
Labels:	area-AssemblyLoader-coreclr, area-Interop-coreclr, untriaged
Milestone:	-

[AaronRobinsonMSFT]

This looks like an issue with discovering assemblies in a COM scenario. The COM loading itself doesn't prescribe how assemblies are placed on disk but does generate a SxS manifest when compiling a COM sever with EnableRegFreeCom. I think this would require updating one of the JSON configuration files to be made aware of the assembly layout on disk.

@vitek-karas or @elinor-fung do either of you have thoughts on how one could educate the loader or update the appropriate JSON file for finding COM components in a subfolder?

[vitek-karas]

I'm guessing that the .dlls in question are all managed assemblies, right? And I also assume that they're not referenced by the application project in any way (.csproj)?

The simplest way to do this would probably be in code. Register an event handler for AssemblyLoadContext.Default.Resolving event and in it implement whatever custom resolution of managed assemblies is needed. The code should find the right file on disk and load it via AssemblyLoadContext.Default.LoadFromAssemblyPath.

This should fix it for any errors which are due to the application failing to find an assembly.

[rlktradewright]

[Sorry for the delay in responding to this, and the length of this post, but this stuff is tricky and important...]

Yes, the .dlls are COM-interop runtime callable wrappers (RCWs), generated by explicit use of tlbimp.exe or aximp.exe. @vitek-karas's assumption that 'they're not referenced by the application project in any way' is completely incorrect: the interop libraries have to be referenced by the project to make any sensible use of the underlying COM libraries.

And for this reason, the suggestion of making the code explicitly load these dlls is quite simply a non-starter. It needs to work in the straightforward fashion that it has for years in .Net Framework.

I've attached two simple Visual Studio projects that demonstrate the issue: one using .Net Framework that works as intended, and another with identical code but using .Net 5.0 that doesn't. These projects use the Microsoft Scripting Runtime COM library as a convenient simple test case. (See the section Sample Apps below).

But before I go any further, let me say what does work in .Net 5: if the COM library is referenced directly via the COM component list in Visual Studio, and Embed interop Types and Copy Local are not both set to No, then everything works fine. The RCW is then either embedded in the executable or is in the same folder as the executable, so no need to find it at runtime, and everything is very straightforward.

Also I can confirm that using <probing privatePath="..." /> to locate .Net assemblies does work correctly. It is only these COM-interop RCWs that are not found. I presume this indicates that the .Net 5 runtime is using a different algorithm when attempting to locate RCWs.

Background and current .Net Framework approach

The simple scenario that works described above is very different from my scenario. Here we have a product that was originally developed as a set of Visual Basic ActiveX dlls and control libraries, together with a set of .exes. This product can be used as a platform for third-party developers to produce their own applications. Some of the .dlls have since been ported to .Net Framework, and some .Net applications have been developed using a mixture of the COM and .Net components.

So the current situation is that the consumers of the COM components may be:

• native .exes (created using any COM-compliant technology, such as Visual Basic 6, Delphi, or even Excel/Word/Visio etc)
• other COM components (the structure is very heavily modularised and layered)
• .Net dlls
• .Net exes,

When deployed, none of the COM components are registered on the target computers, since I use registration-free COM throughout - though of course on development machines at least the ActiveX Control components have to be registered.

This has led to the following folder structure which has served me well for years:

bin ---|                    (All the .exes. These have application manifests that enable the
       |                    lower-level components to be located)
       |
       |--- Platform        (25 domain-specific VB6 .dlls and .ocxs together with an assembly
       |                    manifest for registration-free COM)
       |
       |--- Platform.Net    (.Net ports of some of the corresponding COM components in the
       |                    Platform folder)
       |
       |--- Common          (10 general purpose VB6 .dls and .ocxs, one type library for
       |                    Win32 API access, and an assembly manifest)
       |
       |--- Common.Net      (.Net ports of some of the corresponding COM components in the
       |                    Common folder)
       |
       |--- Providers       (7 plug-in service providers and an assembly manifest)
       |
       |--- Externals       (11 Microsoft redistributable .dlls and .ocxs and an assembly
       |                    manifest - these are not used by .Net .exes)
       |
       |--- Interop         (68 RCWs generated during the build using tlbimp.exe and aximp.exe)

The .Net exes in the bin folder locate their dependencies using the <probing privatePath="..." /> element in their app.config files, like this:

  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <probing privatePath="Platform;Platform.Net;Common;Common.Net;Providers;Interop" />
    </assemblyBinding>
  </runtime>

I very much want to preserve this structure going forward with .Net 5, but at present this is not possible because the Interop folder in the <probing ... /> elements seem to be ignored.

Sample Apps

The attached samples contain a trivial single-form Windows Forms app: one version for .Net Framework, and one for .Net 5. The apps use the Microsoft Scripting Runtime component to enumerate the computer's drives and write their labels to a textbox.

TestDotNetFramework.zip
TestDotNet5.zip

In both apps, the Interop.Scripting dependency has both Embed interop Types and Copy Local set to No. The Interop.Scripting.dll RCW is located in the Interop folder, and the Interop folder is named in the <probing ... /> element in app.config.

The .Net Framework version runs just fine, but the .Net 5 version fails miserably. What is particularly odd is that using fuslogvw.exe, nothing is recorded at all for the .Net 5 version - it evidently never gets as far as even attempting any assembly resolution.

On the other hand, change the .Net 5 app so that either Embed interop Types or Copy Local is set to Yes, and everything is fine.

Other Considerations

I understand that a lot of effort has been put into improving the various sorts of interop in .Net 5, and it may well be that Embed interop Types = Yes is now the 'one true way' as far as COM interop is concerned. However there is a complete lack of any documentation regarding this that I have been able to find - please someone enlighten me if I've missed it.

Using Embed interop Types = Yes in the .Net Framework has never worked for me, because many libraries want to use the same COM types and Visual Studio then complains vociferously about this when more than one such library is used in a project.

The structure outlined above avoids such problems because the RCWs are built once, stored once in a well-defined location, and referenced on an as-needed basis by individual projects. That's why I want to be able to keep it, unless someone can confirm that 'Embed Interop Types` has now somehow solved these issues.

[elinor-fung]

Thanks for the sample repro.

assumption that 'they're not referenced by the application project in any way'

I think perhaps the underlying question there is if they are referenced in such a way that they will be included in the application's deps.json file, which is used for locating dependencies. As you called out, they are definitely referenced by the application. But when the ComReference has Copy Local set to No (Private=false in the .csproj), those dependencies are not included in the .deps.json.

The general issue here seems to be configuring an application to use a subdirectory for assembly loading - similar to dotnet/sdk#10366.

In this case, since the dependencies are already not part of the .deps.json, I'd agree with @vitek-karas that the simplest way would be to have a handler for AssemblyLoadContext.Default.Resolving (called when the default resolution done by the runtime doesn't find the assembly). It is definitely not as simple as having <probing.../>, but the handler could work in conjunction with a .runtimeconfig.json setting, such that the specific subdirectories that the handler looks in could be set through the config file. For example:

static void Main()
{
    AssemblyLoadContext.Default.Resolving += ResolveAssembly;
    ...
}
private static Assembly ResolveAssembly(AssemblyLoadContext alc, AssemblyName assemblyName)
{
    string probeSetting = AppContext.GetData("SubdirectoriesToProbe") as string;
    if (string.IsNullOrEmpty(probeSetting))
        return null;

    foreach (string subdirectory in probeSetting.Split(';'))
    {
        string pathMaybe = Path.Combine(AppContext.BaseDirectory, subdirectory, $"{assemblyName.Name}.dll");
        if (File.Exists(pathMaybe))
            return alc.LoadFromAssemblyPath(pathMaybe);
    }

    return null;
}

The config property could be set in the .csproj (which would be propagated to the generated .runtimeconfig.json):

<RuntimeHostConfigurationOption Include="SubdirectoriesToProbe" Value="Platform;Interop" />

As you said, this would obviously not be as straightforward as the config in .NET Framework, but unfortunately, I don't know that there is currently a simple way to do this purely through configuration files in .NET 5.

[rlktradewright]

@elinor-fung thanks for the comprehensive answer.

Your proposal is pretty simple and works fine in my simple sample app.

How it would scale for the scenario I outlined before, is not clear to me at present, but I don't anticipate any major issues apart from the tedium of doing it.

My antipathy towards it is 'philosophical': it means that the deployment layout has to effectively be encoded in the project file for each application, and my feeling is that these two things should be entirely separate. Thus it's fine that the build puts everything into a single folder (more or less), but when there are dozens of libraries and many apps, it makes a lot of sense to be able to apply some sensible structuring when everything is deployed, in a way that's simple and straightforward.

The ability to have all the .exes in a top-level folder and all the dependencies in (a set of) subdirectories seems so straightforward and simple and obvious that I'm surprised it wasn't a design goal for .Net 5 from day 1.

As a prime example of this structure, take a look at Visual Studio itself. Its main program devenv.exe is small, and its devenv.exe.config uses <probing privatePath="..." /> to locate hundreds of megabytes of further stuff.

So I guess Microsoft aren't planning on porting Visual Studio to .Net 5 any time soon... Shame really, dogfood and all that!

[elinor-fung]

It is also possible to use a template runtimeconfig.json file instead of putting the settings in the project file. By default, the build will look for a file named runtimeconfig.template.json in the same folder as the project - the UserRuntimeConfig project property can be set to point to a different file.

[vitek-karas]

In theory the .deps.json is basically the configuration file and can be used to achieve similar results as the privatePath in .NET Framework. The problem is that it's very detailed (per-assembly as oppose to per-directory) and thus much harder to modify maintain.