Native library packaging to support platforms with restricted file-system access

[timyhac]

The way libplctag.NativeImport packages the runtimes is not standard. NativeImport extracts the appropriate native library to disk at runtime.

It was originally done this way to facilitate compliance with libplctag's LGPL license (requiring a way for end users to substitute the native binary). Now libplctag is dual licensed with MPL so this is not required.

This is a problem because some applications can not write into the application directory (UWP apps). As far as I understand, this is a security feature. Snaps (snapcraft.io) also feature an immutable application directory, so the technique of extracting the library at runtime won't work.

Most .NET packages that ship native libraries ship them in a way that the native library is extracted as part of the build process (rather than at runtime). See https://dev.to/jeikabu/nupkg-containing-native-libraries-1576

@timyhac 's expertise in developing nuget packages is limited, and the non-standard way was selected because it wasn't straightforward how to do it the standard way but still keep maximum compatibility:

• Absolutely need example of how to include native libraries NuGet/Home#6648
• Guide for packaging C# library using P/Invoke to per-architecture and/or per-platform C++ native DLLs NuGet/Home#8623

A limitation of the "extract at runtime" procedure is that applications require permissions to write to disk. Depending on the application, this may not be practical (e.g. all UWP applications have highly restricted access to filesystems). If an application requires file system access purely for this feature, it would be an unusual experience for end-users "why does this need access to disk? I can understand why it needs access to local network, but what files is it accessing?!"

The requirements for the "pre-packaged native library" feature are:

• Support any .NET runtime that can run .NET Standard 2.0
  • linux/windows/macos/iOS/Android/etc..
  • both .NET Core and .NET Framework
• Support for over-riding the binary (in case of testing a pre-release or other customised binary)
• Support for platforms where the binary is not packaged (e.g. ARM)
• Support as a referenced package (i.e. should exhibit this behaviour whether it is directly referenced, or indirectly as part of primary libplctag.NET package)
• Updating the nuget package also updates the native library.

Update 27/6/2024: Microsoft has released some guidance on including Native files into packages, including for .NET Framework.

[timyhac]

Doing some research into this again, there seem to be a few approaches to this.

• Embeddeding the unmanaged libraries as a resource into the managed library, extracting only the correct one to disk, then PInvoke loads it to memory from disk.
  • There does appear to be ways at loading the library from specific directories, so there may be some scope to extract the library to somewhere that the application will have permissions.
  • https://spin.atomicobject.com/2019/10/21/nuget-package-native-dll/
  • csproj: document how to properly pack platform-specific native assemblies NuGet/Home#6645
• Embedding the unmanaged library as a resource, and then directly loading to memory, i.e. without writing to disk first.
  • https://github.com/dretax/DynamicDllLoader
• Using the nuget package runtimes/{RID}/native/plctag.dll technique. This means that the deployed application will be platform specific (i.e. the application developer can't use AnyCPU when preparing a build), but it will also mean that the application will as small as it can be because it doesn't contain the full suite of unmanaged libraries.
  • https://docs.microsoft.com/en-us/nuget/guides/create-uwp-packages
  • https://docs.microsoft.com/en-us/nuget/create-packages/supporting-multiple-target-frameworks

Although this doesn't directly solve the issue at hand, there are some other architectural choice that could be made:

• Providing the native API interface in one package, and then shipping different "providers" for each platform. The major SQLite libraries make use of SQLitePCL.raw. I'm not sure how the platform provider is resolved, but I guess it happens at build time.
• More exotic: compiling the C library to be a .NET managed library.
  • https://ericsink.com/entries/sqlite_llama_preview.html
  • https://www.codeproject.com/Articles/1128868/Compiling-Your-C-Code-to-NET-Part
  • https://dotnetfoundation.org/blog/2015/04/14/announcing-llilc-llvm-for-dotnet
  • https://github.com/praeclarum/Iril

Update: In fact, this article by Eric Sink details the problem perfectly, and in fact this was the article that lead to the current solution: https://ericsink.com/entries/native_library.html

The difference now however is that .NET 5 has been released. Unfortunately, those APIs are not available in .NET Standard 2.0.

[GitHubDragonFly]

In your LibraryExtractor.cs file, you are using System.Uri to detect the directory and none of the System.Uri constructors seems to be correctly handling certain paths with certain signs in them.

Instead, you should just try to detect the current directory like this:

        var extractDirectory = "." + System.IO.Path.DirectorySeparatorChar;

which should work for Windows and Linux (and hopefully other OS).

Also, all your plctag libraries within the "runtime" folder are pointers to github location.
If somebody downloads your project as a zip file will not be able to use it as such.

[timyhac]

Thanks @GitHubDragonFly - can you give an example of the kind of path where its not working by any chance?

This issue is more around sandboxed application types where you can't modify the application directory after installation. UWP fits into this category, there is no concept of a Current Working Directory in UWP. Same with snaps. I think Android and iOS apps also fit this.

P.S. we use LFS to store the native libraries. If you do git lfs clone https://github.com/libplctag/libplctag.NET.git you should get the full project including those files.

[GitHubDragonFly]

My project has "#" in the name and that's where uri escapes and directs extraction of the library to the parent folder of the project instead of the Debug folder where the application is.

I used the libraries from the latest libplctag release and all these files in your project amount to 2.2 MB.
Some people might opt to go with a zip file instead of clone and that's when they will realize that it doesn't work properly.

[GitHubDragonFly]

Have you tried using this: var extractDirectory = Directory.GetCurrentDirectory();

[timyhac]

Another reference example that could be used: https://github.com/mono/SkiaSharp/tree/main/nuget

[juliankock]

Hey!

I just stumbled upon an issue with a constraint container environment (read-only filesystem) so I wanted to see what I can do related to this. I managed to do something like this;

<ItemGroup>
    <None Update="runtimes\linux\arm\libplctag.so" Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsLinuxArm) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\linux\arm64\libplctag.so" Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsLinuxArm64) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\linux\x64\libplctag.so" Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsLinuxX64) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\linux\x86\libplctag.so" Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsLinuxX86) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\osx\arm64\libplctag.dylib" Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsOsxArm64) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\osx\x64\libplctag.dylib" Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsOsxX64) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\win\arm\plctag.dll" Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsWindowsArm) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\win\arm64\plctag.dll"  Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsWindowsArm64) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\win\x64\plctag.dll"  Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsWindowsX64) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="runtimes\win\x86\plctag.dll"  Link="%(FileName)%(Extension)">
      <CopyToOutputDirectory Condition="$(IsWindowsX86) == true">PreserveNewest</CopyToOutputDirectory>
    </None>
  </ItemGroup>

  <PropertyGroup>
    <IsLinuxArm Condition="$([MSBuild]::IsOsPlatform('Linux')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == Arm">true</IsLinuxArm>
    <IsLinuxArm64 Condition="$([MSBuild]::IsOsPlatform('Linux')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == Arm64">true</IsLinuxArm64>
    <IsLinuxX64 Condition="$([MSBuild]::IsOsPlatform('Linux')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == X64">true</IsLinuxX64>
    <IsLinuxX86 Condition="$([MSBuild]::IsOsPlatform('Linux')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == X86">true</IsLinuxX86>
    
    <IsOsxX64 Condition="$([MSBuild]::IsOsPlatform('OSX')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == X64">true</IsOsxX64>
    <IsOsxArm64 Condition="$([MSBuild]::IsOsPlatform('OSX')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == Arm64">true</IsOsxArm64>

    <IsWindowsArm Condition="$([MSBuild]::IsOsPlatform('Windows')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == Arm">true</IsWindowsArm>
    <IsWindowsArm64 Condition="$([MSBuild]::IsOsPlatform('Windows')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == Arm64">true</IsWindowsArm64>
    <IsWindowsX64 Condition="$([MSBuild]::IsOsPlatform('Windows')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == X64">true</IsWindowsX64>
    <IsWindowsX86 Condition="$([MSBuild]::IsOsPlatform('Windows')) And $([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture) == X86">true</IsWindowsX86>
  </PropertyGroup>

I don't know if this is available for .NET Framework.

I would really appreciate it if it was already packed into the NuGet package. Maybe we/I could revive #249

[timyhac]

@juliankock - thanks for bringing this up!

I've been meaning to get around to having another crack at this issue and was thinking we may need to multi-target.

[timyhac]

@juliankock - I've just now re-tested what the code in the PR looks like and it seems to work for .NET Core but not for .NET Framework.. Geez it is such a hassle testing this.

The good news is that it looks like Microsoft has released some guidance on what to do here!
https://learn.microsoft.com/en-us/nuget/create-packages/native-files-in-net-packages

[timyhac]

@juliankock - I've managed to get something working that seems to provide the better experience for .NET Core applications but the same experience for .NET Framework and other .NET platforms. I couldn't figure out how to do it the "right" way (according to that linked document) for .NET Framework. It is terribly ugly, introduces an additional 4 packages, and I'm not terribly confident in the design.

I would release alpha package(s) but creating those other packages is a problem if this doesn't end up being the right design - I don't want to be stuck supporting them.

If there was a separate nuget feed where we could do some experimentation that could work. I have seen Microsoft use some service for this where they wanted to release a package for feedback but be 100% sure that only people that really wanted to experiment would go through the hassle of setting up the separate nuget feed.

[timyhac]

I have been working on this and believe that we're about 90% of the way there. I have recently been focused on the nightly/preelease builds issue.

• Requested an account with MyGet.org to host nightly/prerelease builds. Using MyGet seems to be what other players in the .NET ecosystem do.
• Have experimented with Github packages for the same purpose - hosting nightly/prerelease builds. Although this works, it could actually be a problem that Github packages feature integrates so well with the rest of Github - I don't want users to be confused about which package and nuget feed to use.
• Developed a new build system that will more easily accommodate new scenarios such as this.

While working through the ramifications of changing how we ship libplctag core binaries, I found a few scenarios that will need to be thought through more deeply:

• Single-file deployments - I have not attempted to get this working, but I imagine that shipping native binaries inside a Single-file deployment is difficult. I believe this does work with the current package because the native binaries are extracted at runtime.
• Other .NET platforms that are supported by netstandard2.0 but are not one of the main .net runtimes (net5.0+, netcore, .netframework47).
• Using LGPL as a license. One of the original design goals for libplctag.NativeImport was to support end users being able to use their own native binaries as is required by LGPL. Now that libplctag.NativeImport elects to use the MGPL2 license this isn't a problem - and swapping out the native binaries is still going to be possible (and likely much easier), but it will require some additional instructions for end-users.

[timyhac]

@juliankock - I've released an alpha version of the libplctag.NativeImport package which uses Nuget/MSBuild's runtimes feature for NetCore projects. It would be great if you could test it and provide some feedback (please include as much info as possible about your target platform).

[juliankock]

@timyhac - We now have a docker container running with the pre-release, mimicking a production workload. I will observe if we experience any problems, but currently it connects fine to the PLC. The current docker container runs on a Linux host, and the docker container is built for linux-x64, built with https://hub.docker.com/r/microsoft/dotnet-sdk version 8.0 and running on https://hub.docker.com/r/microsoft/dotnet-aspnet/ 8.0.

I will keep you posted. Thank you so much for the fast handling of this issue.

[timyhac]

Brilliant - that is good info, thanks!

I'm going to mark this as closed as the intent of the fix has been committed and merged - if there are problems, please raise a new ticket.

This is great, it was a bit like unfinished business - good to get it over the line.