API Set Usage Question

[wjk]

I am trying to write a complex application (private repo) that uses CoreCLR. Since I want compatibility back to Windows 7, I must use the Microsoft.NETCore.Windows.APISets NuGet package to ensure that the application will run. However, I am running into a problem.

Having 120+ DLLs in the same directory from this one package alone (not counting my app itself, as well as the CoreFX packages I use) doesn't quite work for me, though. Therefore, I have chosen to place the CoreCLR DLLs, as well as the API Set stub DLLs, in a subdirectory. Unfortunately, once I do so they then are not found at runtime, and therefore my app won't launch.

I have tried using SxS modules, only to find that LoadLibraryEx() isn't aware of them AFAIK. I have then tried using AddDllDirectory() (with a fallback to manual %PATH% modification if that routine isn't available), only to find that neither of them will get my app to find coreclr.dll. Is there any reason why calling AddDllDirectory() is not having any effect upon DLL loading, contrary to what MSDN says? (I am loading coreclr.dll using just its base file name, calling SetDllDirectories(), as well as using the LOAD_LIBRARY_SEARCH_USER_DIRS flag to LoadLibraryEx(), if it helps.) Is there another technique that will allow me to put the DLLs in a subdirectory, I want to be sure that P/Invoke will be able to find the DLLs, as I use a lot of P/Invoke in my application.

Apologies if this isn't the correct place to ask this.

[wjk]

Update: If I reference coreclr.dll though an SxS assembly, and I load that file by name instead of its full path, the DLL loads correctly. However, ICLRRuntimeHost2::Start() then returns E_FAIL. My first guess is that one of the support DLLs used by CoreCLR (mscordbi.dll for example) is not being loaded correctly. I have listed all of these DLLs in my SxS manifest, however.

The problem with the E_FAIL code is that it doesn't tell you what went wrong, only that something did. @wtgodbe @sergiy-k Can you give me any pointers?

[wjk]

And now I am getting the error code 0x80131022 ("Invalid operation" according to Visual Studio). I really have a hard time wrapping my head around CoreCLR's inner workings, so I cannot diagnose it any further.

[am11]

Is there any reason why calling AddDllDirectory() is not having any effect upon DLL loading, contrary to what MSDN says?

We recently ran into a similar problem with DllImport, when transforming a CLS project to P/Invoke (FFI) usage sass/libsass-net#34. Windows (v10, all updates applied) loads wrong native lib even after specifying the absolute path by pinvoking LoadLibraryEx/AddDllDirectory et al. We had the native DLL installed in %ProgramFiles%, and although LoadLibraryEx specified a separate path $(OutputDir)/x64/libsass.dll (native DLL in subdirectory of project's $(OutputDir)), it was picking the DLL from %ProgramFiles%. Read library loading precedence ordering on MSDN again, made sure %ProgramFiles% copy is not already loaded in memory before hand but still could not get it to load the correct lib. Renaming the DLL in ProgramFiles resulted in loading the correct DLL in subdirectory.

For debugging, first used DependencyWalker and WinDBG to find out which modules the consumer console app is trying to load. DepnedencyWalker did not helped much as it resulted In some module load failures. WinDBG was also not very clear on mapping managed<->native libs. Finally got an exact path of dll it was loaded using ProcessExplorer:

• Open solution in VS > add breakpoint just before where the library load happens and run the startup project.
• Download and open ProcessExplorer > Ctrl+F > search your startup project executable > expand the related nodes in list.
• Advance the debugger and step through the line where it loads the native library.

At this point ProcessExplorer will reveal the native library the process has just loaded (by highlighting/flashing it in the expanded list).

We came to conclusion that this is a Windows bug, although it is very hard to determine when there are so many precedence rules involved. So as a workaround, we ended up having two MSBuild tasks in .csproj file to emit x86/x64 versions from external .sln (residing in submodule directory). It is still work in progress but for packaging, we have decided to have NuGet package DLLs for x64 and x86 such that it would lay down the correct package on install target (https://docs.nuget.org/consume/projectjson-format#runtimes). For local testing, we have a copy task to get x64 version from $(OutputDir)\x64\ to $(OutputDir)\ (fixed architecture, as the managed lib builds with AnyCPU and runtime process introspection to get actual bit-ness was not a suitable option).

This is not the ideal solution. As the consumer of lib may change their the target architecture of their project after restoring the package and NuGet will not detect that the restored package is invalidated.

We may also try unloading the library that is loaded in the memory before engaging LoadLibraryEx, as the initial plan was to package all architecture variants libs for all targets (but haven't tested it so far).

HIH in some way. :)

[wjk]

@am11 I actually switched back to using SxS modules due to the reason you specified. I thought that LoadLibraryEx() is unaware of SxS modules, but the real culprit was the fact that I was giving an absolute path. LoadLibraryEx() knows to search SxS modules when given a base name only.

I have also copied mscorlib.dll into the same directory as CoreCLR and added a <file/> element to my SxS manifest so that Windows knows of its existence; that seems to help a little.

The current problem is that CoreCLR is being loaded correctly, but ICLRRuntimeHost2::Start() is returning the "Invalid operation" error.

[wjk]

As it turns out, I am now suspecting that the SxS modules have nothing to do with what is going on here. Having commented-out the SxS <dependency> element in my manifest, and copied the DLLs into the same directory as the EXE, I still get HOST_E_INVALIDOPERATION (0x80131022). I suspect that something else in CoreCLR has changed, but have no idea what.

[wjk]

Closing in favor of dotnet/coreclr#3047, which illustrates the problem I am currently having in a clearer way.