[NativeAOT] Building native app against NativeAOT-built static library creates duplicate LNK2005 errors (already defined in)

[darkguy2008]

So it's pretty late right now but I could try to set up a repo tomorrow.

Basically I'm getting an issue where I'm building a native library using the suggestions in the docs:

csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="7.0.0-*" />
  </ItemGroup>

</Project>

Class1.cs:

using System;
using System.Runtime.InteropServices;

namespace NativeLibrary
{
    public class Class1
    {
        [UnmanagedCallersOnly(EntryPoint = "SetData")]
        public static int SetData(int a, int b)
        {
            return a + b;
        }
    }
}

Nothing too fancy, compiled with:

dotnet publish /p:NativeLib=Static --self-contained -r win-x64 -c release

Note the required use of --self-contained. If I don't, I get an error, so I'm basically forced to include the .NET runtime in the static lib.

Alright, so I go and create my native app including the native lib I just created, nothing too fancy, again:

csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net6.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="7.0.0-*" />
  </ItemGroup>

  <ItemGroup>
    <DirectPInvoke Include="NativeLibrary" />
    <NativeLibrary Include="<Path to your publish dir>\NativeLibrary.lib" />
  </ItemGroup>
    
</Project>

Program.cs:

using System;
using System.Runtime.InteropServices;

namespace Native
{

    public class Program
    {
        [DllImport(@"NativeLibrary")] public static extern int SetData(int number1, int number2);

        static void Main(string[] args)
        {
            Console.WriteLine($"Test! {SetData(100, 200)}");
        }
    }

}

Compiled with:

dotnet publish -r win-x64 --self-contained -c release

Then I get a huge bunch of compiler errors, here's an excerpt of it when it ends (it's just a whole lot of repeated warnings (treated as errors) mostly related, I assume, to having the .NET self-contained framework statically linked in the native library and then attempting to link against it:

NativeLibrary.lib(NativeLibrary.obj) : error LNK2005: _unwind0S_P_Reflection_Core_System_Reflection_Runtime_MethodInfos_CustomMethodInvokerAction__InvokeOpenInstanceThunk already defined in Native.obj [C:\Users\User\Desktop\Test2\Native\Native.csproj]
NativeLibrary.lib(NativeLibrary.obj) : error LNK2005: _unwind0S_P_CoreLib_System_Runtime_InteropServices_DllImportResolver__InvokeOpenInstanceThunk already defined in Native.obj [C:\Users\User\Desktop\Test2\Native\Native.csproj]
NativeLibrary.lib(NativeLibrary.obj) : error LNK2005: _unwind0S_P_CoreLib_System_Reflection_MemberFilter__InvokeOpenInstanceThunk already defined in Native.obj [C:\Users\User\Desktop\Test2\Native\Native.csproj]
NativeLibrary.lib(NativeLibrary.obj) : error LNK2005: _unwind0S_P_CoreLib_System_Reflection_TypeFilter__InvokeOpenInstanceThunk already defined in Native.obj [C:\Users\User\Desktop\Test2\Native\Native.csproj]
NativeLibrary.lib(NativeLibrary.obj) : error LNK2005: _unwind0S_P_CoreLib_System_DateTimeParse_MatchNumberDelegate__InvokeOpenInstanceThunk already defined in Native.obj [C:\Users\User\Desktop\Test2\Native\Native.csproj]
NativeLibrary.lib(NativeLibrary.obj) : error LNK2005: _unwind0S_P_CoreLib_Interop_Advapi32_EtwEnableCallback__InvokeOpenInstanceThunk already defined in Native.obj [C:\Users\User\Desktop\Test2\Native\Native.csproj]
NativeLibrary.lib(NativeLibrary.obj) : error LNK2005: _unwind0Internal_CompilerGenerated__Module___InvokeRetV already defined in Native.obj [C:\Users\User\Desktop\Test2\Native\Native.csproj]
     Creating library bin\release\net6.0\win-x64\native\Native.lib and object bin\release\net6.0\win-x64\native\Native.exp
bin\release\net6.0\win-x64\native\Native.exe : fatal error LNK1169: one or more multiply defined symbols found [C:\Users\User\Desktop\Test2\Native\Native.csproj]
C:\Users\User\.nuget\packages\microsoft.dotnet.ilcompiler\7.0.0-alpha.1.22058.1\build\Microsoft.NETCore.Native.targets(364,5): error MSB3073: The command ""C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64\link.exe"  @"obj\release\net6.0\win-x64\native\link.rsp"" exited with code 1169. [C:\Users\User\Desktop\Test2\Native\Native.csproj]

---

Any ideas what could we do here? Maybe try to avoid forcing --self-contained for static libs? My goal is to create a single-file binary with the native library statically linked (therefore, no extra DLL next to the executable file) but I'm unable to :(

---

Edit: I tried adding these to the .csproj, but they didn't do anything:

    <WarningsAsErrors>false</WarningsAsErrors>
    <MSBuildWarningsAsErrors>false</MSBuildWarningsAsErrors>
    <MSBuildTreatWarningsAsErrors>false</MSBuildTreatWarningsAsErrors>
    <NoWarn>$(NoWarn);LNK2005</NoWarn>

I also added this, which ignored the warnings/errors, but then it crashed with another one:

  <ItemGroup>
    <LinkerArg Include="/FORCE:MULTIPLE" />
  </ItemGroup>

     Creating library bin\release\net6.0\win-x64\native\Native.lib and object bin\release\net6.0\win-x64\native\Native.exp
bin\release\net6.0\win-x64\native\Native.exe : warning LNK4088: image being generated due to /FORCE option; image may not run [C:\Users\User\Desktop\Test2\Native\Native.csproj]
NativeLibrary.lib(NativeLibrary.obj) : fatal error LNK1223: invalid or corrupt file: file contains invalid .pdata contributions [C:\Users\User\Desktop\Test2\Native\Native.csproj]
C:\Users\User\.nuget\packages\microsoft.dotnet.ilcompiler\7.0.0-alpha.1.22058.1\build\Microsoft.NETCore.Native.targets(364,5): error MSB3073: The command ""C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64\link.exe"  @"obj\release\net6.0\win-x64\native\link.rsp"" exited with code 1223. [C:\Users\User\Desktop\Test2\Native\Native.csproj]

Any pointers welcome, thanks!

[MichalStrehovsky]

Why do you need to include the NativeAOT compiled library from your EXE?

If you reference the library as a normal assembly reference (ProjectReference, or PackageReference or any other mechanism like that), the compiler is going to compile it into native code for you as part of native compiling the EXE.

[darkguy2008]

Hey! Thanks for your reply :)

Well yeah I could add the library as a project reference, assuming it's also compatible with NativeAOT (it is, for now). I think I was too tired yesterday to think about that, I was focused in trying to get a NativeAOT app link another NativeAOT library inside.

Actually I think the issue is different - How would I include native libraries in the same binary to get a single-file build? Just like these properties do for non-AOT builds:

    <IncludeAllContentForSelfExtract>true</IncludeAllContentForSelfExtract>
    <IncludeNativeLibrariesInSingleFile>true</IncludeNativeLibrariesInSingleFile>
    <IncludeNativeLibrariesForSelfExtract>true</IncludeNativeLibrariesForSelfExtract>

I was linking it statically because I'm calling it with P/Invoke, so I was using the suggested idea of DirectPInvoke and NativeLibrary properties, but it seems it doesn't like to include NativeAOT-compiled libraries, only those that don't have the .NET runtime inside.

What if someone else just publishes the LIB file? Without any kind of source code so you can't add it as a project reference?

Or what if someone else publishes a DLL built with NativeAOT, is there a way it can be bundled/included within the build to get a single-file executable, without affecting the linking?

[MichalStrehovsky]

IncludeNativeLibrariesInSingleFile and friends don't work with NativeAOT. NativeAOT works more like other AOT compiled languages (C/C++/Rust, etc.): if you want to include foreign code in your executable, you have to use a static library. .NET Single file (without NativeAOT) currently doesn't have a way to include static libraries, so extracting dynamic libraries is the only way to get a single file there. It has a lot of problems.

You can emulate IncludeNativeLibrariesInSingleFile if you really want - just include the library as an embedded resource and extract it to disk before the first DllImport runs. You might need to use to NativeLibrary.SetDllImportResolver API to redirect DLL lookup to the location where you extracted the library. As you're implementing it, you'll see why it's very problematic (where do we extract? who deletes the file? etc.).

Or what if someone else publishes a DLL built with NativeAOT, is there a way it can be bundled/included within the build to get a single-file executable, without affecting the linking?

You could use a tool like objcopy to rename the symbols in the file. It's what native developers do when they're in such situation. Redistributing static libraries is problematic. It's what dynamic libraries are built for.

[darkguy2008]

Hi @MichalStrehovsky

Thanks for your reply, I've been working on this and so far I think it can be closed. Your comment/expertise lead me to a working solution, inspired by it.

What I ended up doing was creating a main project as the NativeAOT executable, then adding a Managed project as reference. The Managed project would do the DLLimport calls and all that. On the NativeAOT side, I used DirectPInvoke and NativeLib to add a new static library I created as suggested (and as a side effect, it native-compiled the managed reference :) ), but I also had to change some flags, from Multi-Threaded DLL to Multi-Threaded, and disable /LTCG or something like that. This got rid of the duplicate symbols as now a single runtime is added.

You were right in the fact that a NativeAOT library can't include another NativeAOT-compiled library because they contain the full .NET runtime each. The only solution is to make a wrapper in C++ or something else that doesn't include a .NET runtime, in other words, there can be only a single one.

My solution isn't very pretty, and I would love if WebView2 and Windows Forms would finally get some complicance with NativeAOT, but it works for now and I'm happy.

I tried to use objcopy but it threw errors about unable to find the file format or something. I guess you meant it as a solution for Linux? I couldn't get it to do it on Windows. In any case, this issue/question can be closed, although I think it would be a good caveat to add in the documentation.

Thank you!

[kant2002]

@darkguy2008 what problems are you having with WinForms and what kind of compliance do you need? Given that major PR which unlock ComWrappers work in WinForms for me lands, you may expect some progress in that area (only easy targets for now)

[darkguy2008]

Hey @kant2002 ! Thanks for your reply, I've seen you around lately while working on my app going through all these NativeAOT issues I've been having :)

Well, the problems I have with WinForms and NativeAOT mostly are:

• Current Microsoft.DotNet.ILCompiler package equal or greater than 7.0.0-alpha.1.22074.1 has an interesting issue where:
  • It throws an error point blank if UseWindowsForms is true, saying it can't compile the app because Windows Forms isn't available for NativeAOT, blocking the entire compilation step. I honestly think that some of the conditional blockers that the ILCompiler package adds to the build process aren't very helpful, it should try to compile regardless.
  • PublishTrimmed can't be set to false - This is important to make WindowsForms app work.
  • In a WinForms app, if you set PublishTrimmed to false (with a previous version of the IL compiler), it still compiles but generates a big file.
• File size - any WinForms app + NativeAOT ends up being like 30+ Mb or something. Of course, without trimming.
• Any NativeAOT compiled app with Windows Forms ends up with a lot of trim warnings (I've discovered that each trim warning seems to increase file size - it works though)

The first two points in my list may be a bit confusing because I'm writing what I remember from a couple days ago, in the end I've fixed my issues (for now) by using a project as "entrypoint" that includes another as reference and spawns the app from there, where the entrypoint does not have UsesWindowsForms and has PublishTrimmed set to true. This works (and please leave it as-is! The new change in the ILCompiler package made me lose like 3 hours trying to figure out what was wrong and find this solution) but it produces a lot of trim warnings and huge file size, but at least it solves the problem I'm looking to solve by using NativeAOT to generate a native binary that can't be easily decompiled.

I really appreciate all the work that's been done to make WinForms compatible with NativeAOT though, I was able to use your ComWrappers project and it kinda worked in some initial tests, but it didn't have wrappers for WebView2 ( obviously :P ) so I couldn't use it, but I see it working for similar basic apps for now.

If you need more info let me know though, I'll be more than happy to help improve this area :)

[kant2002]

@darkguy2008 Thank you for your kind words. I have to admit that NativeAOT experience far from perfect, I want to redirect some blame as not applicable to NativeAOT (actually only first one not applicable)

1. UseWindowsForms + PublishTrimmed=true gives warning. This is issue which affects NativeAOT obviously, but in same way it is blocks ReadyToRun. Warning which you are seeing was introduced Show trimmer warnings for WinForms and WPF sdk#18617 and if you follow PR you will notice that adding <_SuppressWinFormsTrimError>True</_SuppressWinFormsTrimError> to your PropertyGroup section will unblock you (Obviously this is undocumented and as such unsupported). For reference that was introduced in .NET 6 SDK so when you install it, you break ILCompiler.
2. File size - any WinForms app + NativeAOT. I did not expect too much very soon on this front. I suspect that class hierarchy make this a problem. You can troubleshoot what cause code to be included if you follow this guide https://github.com/dotnet/runtime/blob/main/docs/workflow/debugging/coreclr/debuging-compiler-dependency-analysis.md . I would not expect that troubleshooting be too pleasant experience. If you have any interesting findings please share, that may allow me change directions for some work.
3. Any NativeAOT compiled app with Windows Forms ends up with a lot of trim warnings. Here I have only one suggestion, find trim warning, find source code for that spot at https://github.com/dotnet/winforms and contribute fix. Any small fix in that area is aligned with their goals Epic - Make WinForms trim compatible winforms#4649 . And this is reason why you see you main issue with PublishTrimmed=true. Winforms not yet ready for trimming (which runs before ILC) in such way that it is easy to shut yourself in the foot (that's why it is blocked by warning). This is in spirit of my general suggestion about WinForms + NativeAOT - Use locally built version of WinForms.
4. The new change in the ILCompiler package made me lose like 3 hours trying to figure out what was wrong. I would suggest you to join CoreRT Gitter https://gitter.im/dotnet/corert where you will find like-minded guys who work with NativeAOT. made me lose like 3 hours not sure that we will save you that time. Still join.
5. but it didn't have wrappers for WebView2 I think everything is ready to make your own ComWrappers which supports WebView2 (given that I did not convince WebView2 team to work on that). If you feel brave enough, I would welcome such contribution. That would be better discussed in the WinFormsComInterop repo.