EPIC: Docs on creating packages that are wrappers of native code

[zivkan]

This issue is tracking customer requests about how to create a package that is a .NET wrapper of native dll/so.

Out of scope: packages to be used by native (c/c++) projects

• Improve documentation on use of runtimes folder and RIDs #600

• Current approach to packaging x86 and x64 assemblies #1083

• Improve documentation - bait and switch pattern, other #1282

• nuspec with dll, native and content in output folder #2038

• What is win10 TFM and what is the difference with win? #2237

• Include a how-to link for creating a Nuget that references native libraries #2287

• Including native assemblies is very confusion and underdocumented #3091

• Creating a package for native / specialized runtimes Home#3157

• Can't install native package with project.json Home#3958

• Including an unmanaged DLL in build output Home#3970

• csproj: document how to properly pack platform-specific native assemblies Home#6645

• Absolutely need example of how to include native libraries Home#6648

• Docs on creating a Native TFM package Home#7168

• Current approach to packaging x86 and x64 assemblies  Home#7374

• [Question] - Pack Native dependencies Home#7479

• Architecture-specific files folder hierarchy lost (native subfolders) Home#7698

• Guide for packaging C# library using P/Invoke to per-architecture and/or per-platform C++ native DLLs Home#8623

• How to copy non assembly files to output directory of application Home#8843

• nuspec with dll, native and content in output folder Home#8926

• Creating package with x86 and x64 managed and native libraries into single package Home#9272

• csproj: how to pack at build platform-specific native assemblies Home#9361

• Create a x86+x64 .NET 4.6+Native nuget library and it's driving me NUTS Home#9369

• [Question] Packaging Platform specific from Vendor dll Home#9502

• NuGet package's content files cannot be imported to .NET project but can be imported to .NET Framework project  Home#12564

• Common practice to create NuGet package for Windows and Linux Home#11110

• how to pack a project which use pinvoke? Home#12637

• Any CPU is treated as x86 even when it'll really be x64 dotnet/sdk#1545

[tebeco]

This issue is tracking customer requests about how to create a package that is a .NET wrapper of native dll/so.

In order to clarify a bit few points before it's too late, here are few scenario to fully differenciates

Scenario 1

• I am the owner of Native Code (I write the code and can edit/update it to create new release), and i want to properly create and publish nupkg
• I also want to make sure that both .so and .dll and properly packed based on RID
• Consumer should only do dotnet add package and dotnet publish [--runtime RID]

Scenario 2

• I am a Managed code writer, and a Vendor give me RAW DLL/SO
• The vendor REFUSE to make it a package
• I have to pack it myself
• I don't want to use <Reference
• In consuming project I should only do dotnet publish [--runtime RID] and proper raw dll/so should be copied / no msbuild Copy step

Scenario 3

• Everything from Scenario 2
• The vendor ALSO gives us a Wrapper built as a DLL and still refuse to do an nupkg
• This wrapper already contains [DllImport] / DllMap
• As a consumer my project should only do dotnet add package and `dotnet publish
• This means it would requires to create a package with both already built wrapper + native

[wldevries]

Maybe I don't fully understand the scenarios you describe, but it seems to me the following scenario is missing:

• I have a library written in C++/CLI with managed x86 and x64 binaries
• I want to support both packages.config and PackageReference

[tebeco]

@wldevries
This look like this is Scenario 1, edited for clarity, is that better ?

[wldevries]

As I see it there are two types of managed code. There are the native dlls that are treated as content and are used via PInvoke. On the other hand there are the C++/CLI dlls that need to be referenced at compile time. Depending on the type of native dll you need a completely different strategy of including them in NuGet packages.

As of yet I have been able to create packages that support the first type for both package.json and PackageReference, but I have not been able to create working packages for the C++/CLI dlls at all.

[zivkan]

To minimise risk that I forgot again, something similar to this should allow building a c# project to automatically build a c++ project in both 32 and 64. You can then use TargetsForTfmSpecificContentInPackage to pack it into your nupkg. This syntax might not be 100% correct, but it's something like:

<ProjectReference Include=”../whatever/whatever.vcxproj” SetPlatform=”Platform=x86” ReferenceOutputAssembly=”false” />
<ProjectReference Include=”../whatever/whatever.vcxproj” SetPlatform=”Platform=x64” ReferenceOutputAssembly=”false” />

[noamyogev84]

Trying my luck here...
Thanks @tebeco for describing the wanted scenarios. I relate to (3) and until now have no success in accomplishing such a task.
I have 2 netstandard2.0 projects, one of them is DLLImporting an unmanaged x64 dll that needs to be deployed with it in the same folder.

So my solution includes:

1. project 1 [netstandard2.0]
2. project 2 [netstandrd2.0] - have project dependency in project 1, and DLLImporting native.dll
3. native.dll

Building the solution places all the files successfully under bin/debug/netstandard2.0
using dotnet pack/nuget pack just continuously reporting that my native dll is not under lib folder and won't get copied.

It would be really helpful to have a full example of how to accomplish such a task. 🤷‍♂️

[tebeco]

In order to get native properly copied during dotnet publish, remembre that a native is plateform specific
This means that at the "moment" you pack it, you already now that "THIS" binary is for THIS platform.
So you have to "express" this for dotnet pack so that when you will consume and then use/publish that package everything should be copied properly in respective folder

here is an example of what it could/would look like:

https://github.com/tebeco/NativeLibNugetStuff/blob/master/Corp.Common/src/Corp.Common.ThirdPartPackaged/Corp.Common.ThirdPartPackaged.csproj#L9

  <ItemGroup>
    <None Include="third_part_native.dll" Pack="True" PackagePath="runtimes/win-x64/native" />

    <None Include="ThirdPart.ManagedWrapper.dll" Pack="True" PackagePath="runtimes/win-x64/lib/netstandard2.0" />
    <None Include="ThirdPart.ManagedWrapper.dll" Pack="True" PackagePath="lib/netstandard2.0" />
  </ItemGroup>

• As you see this third_part_native.dll is inside a runtimes folder
• the SDK will now knows that it will be related to a dotnet publish --runtime
• win-x64 ... well publish ... you guessed
• and native for native
• now the lib/netstandard2.0 duplicate ... i'm not quite ok in my head, I think I understood and i did not tripple chekc that it might be done for backward compat (eg: other/old way to consume the package)

But this issue has been created for exactly these questions,
@zivkan can you point us to current docs until this epics it completed like:
folder per runtime ? What's the structure ?
is that duplication required and for what purpose, if it's for back compat, do you know where we could find the "old" verison having an issue ?
how do we disable "build output" of a csproj so that we could just run dotnet pack on a csproj that contains only <None ... and they'll be packed / no build done / no "extra dll"
...

[noamyogev84]

KUDOS @tebeco you just solved my problem, after quite some time that i'm working on this. Your explanation and example are perfect, thank you for that! 🎉

[tebeco]

glad it worked out for you ;)
this is why i would love to see a dedicated docs

i'm 110% sure there lots of thing i'm doing wrong ^^

[tebeco]

I wonder if this could be more or less related

dotnet/core#5703
dotnet/core#5699

spotted on https://themesof.net/

[zivkan]

folder per runtime ? What's the structure ?

This is the best (only?) doc we have on the subject at the moment: https://docs.microsoft.com/nuget/create-packages/supporting-multiple-target-frameworks#architecture-specific-folders

is that duplication required and for what purpose, if it's for back compat, do you know where we could find the "old" verison having an issue ?

A few weeks ago someone internally contacted me about creating packages with managed and native dlls. I tried explaining this to them, which they gave me feedback that it helped them a lot:

With PackageReference NuGet selects compile-time and run-time assemblies independently:

• compile time assemblies are selected from ref/, but if none match it falls back to lib/.
• runtime assembles are selected from runtimes/, but if none match it falls back to lib/.
• Therefore, to a PackageReference project, if the package contains lib/, ref/ and runtimes/, all with assets for the same target framework, then the lib/ folder is ignored, it may as well not exist.

With projects using packages.config (and as I discovered this week, Packet, which is common in the F# community):

• only the lib/ assets are used, both for compile time and runtime
• Anything else (runtime assets per runtime identifier) needs to be done by the package author via msbuild props/targets in the package's build/<tfm>/ folder
  • Note that PackageReference imports build/<tfm>/<packageid>.props&targets only when the package when the project has a PackageReference for it. Packages includes as dependencies (trasitive packages) only only import buildTransitive/<tfm>/<packageid>.props&targets. A common pattern I've seen is to have props/targets files in the buildTransitive/... folders that do nothing more than import the props/targets file in build/..., which avoids duplicating contents in the pacakge, if you care about package size.

If you want to support users using both PackageReference and packages.config in their projects, and you want to use the runtimes/ folder, then your props/targets needs to detect when the project uses PackageReference, and skip doing whatever the the props/targets file does, otherwise both NuGet and your own props/targets files will be duplicating the same work. Here's an example: https://github.com/microsoft/MixedReality-WebRTC/pull/634/files

how do we disable "build output" of a csproj so that we could just run dotnet pack on a csproj that contains only <None ... and they'll be packed / no build done / no "extra dll"

This doc has a list of pack input properties: https://docs.microsoft.com/en-us/nuget/reference/msbuild-targets#pack-target-inputs

There are two that appear interesting. IncludeBuildOutput does what you're asking about, it instructs NuGet to not implicitly copy the project's output assembly to the package.

The other is BuildOutputTargetFolder, which appears to tell NuGet where to put the assembly when it does automatically copy the build output. I've never tried it, so I don't know if it has any quirks, but maybe <BuildOutputTargetFolder>lib;runtimes/win-x64</BuildOutputTargetFolder> will cause ThirdPart.ManagedWrapper.dll to be put in lib/netstandard2.0/ThirdPart.ManagedWrapper.dll and runtimes/winx-64/lib/netstandard2.0/ThirdPart.ManagedWrapper.dll.

[tebeco]

nice 👍
I'll take some time to read it carefully since i spotted few mistake I did and suggested to other developer ;)

shame on me

[ssa3512]

I am in scenario 2, vendor provided native dll and so files and we have built a managed wrapper. I am shipping the native wrapper and dlls in an Interop (wrapper) and Native (native assemblies only) packages.

The Native package ships a folder structure that looks like this:

The Interop package depends on the Native package.

I am trying to write some unit tests for the interop wrapper. If I reference my Interop nuget package (PackageReference) the unit test works, but if I reference it via ProjectReference I get a DLL not found error. I tried setting up the Native project to copy the native files as output like this:

This produces the same file and folder structure in my test project output directory on build as when I use PackageReference, but I still get DLL not found.

@tebeco @zivkan do you know of any way to make this scenario work, so the unit test project knows to look in the runtimes/platform/native folder for the current running platform as part of the dll search path when using ProjectReference?

[amaitland]

do you know of any way to make this scenario work, so the unit test project knows to look in the runtimes/platform/native folder for the current running platform as part of the dll search path when using ProjectReference?

@ssa3512 For compile time solution if you add the following to your Unit Test project file, MSBuild will select the current RuntimeIdentifier and generate a build specific to that RuntimeIdentifier (the dll will be copied to your bin folder, you won't get the other folders)

<RuntimeIdentifier Condition="'$(RuntimeIdentifier)' == ''">$(NETCoreSdkRuntimeIdentifier)</RuntimeIdentifier>

[zarenner]

Filed #2322 about unexpected architecture-specific file hierarchy flattening.

[JohnNilsson]

It's also a bit unclear how to package non-assembly files needed by native dll:s

I have a scenario (like the scenario 3 above) where I have a bunch of native dll:s that needs to load additional data to function. I figured the simplest approach to handle was to package those data-file together with the dll:s, but it turns out some tools expect only assemblies in the runtime/ folder. And packaging them as contentFiles won't work due to the block on transitive contentFiles dependencies.

So in addition to how to manage native assmblies please also clarify

• Should non-assemblies be placed under runtime/?
• Should runtime/any/ be used for non-platform specific runtime dependencies?

[zivkan]

Hey everyone, I gave up on trying to create a sample that can be cloned & packed. I also limited the scope of the document to the easiest scenario, otherwise it requires too much effort to document (plus trying to explain it clearly enough). Anyway, it would be great for as many people as possible, especially anyone who successfully created a package already, to review and provide constructive feedback, so we can improve the docs for this advanced scenario:

• .NET package with native libraries #3092

[zivkan]

My PR was perged a while ago and is available on the docs site: https://learn.microsoft.com/nuget/create-packages/native-files-in-net-packages