Referencing external assembly with native dependencies in Azure Function

[dovisio]

I'm getting following exception when doing aforementioned:

2017-12-10T12:56:29.526 Function started (Id=22fce544-1d7f-4262-8c59-dcb962ac0bd1)2017-12-10T12:56:29.574 Exception during runtime resolution of assembly 'Affdex, Version=3.4.1.1320, Culture=neutral, PublicKeyToken=null': 'System.IO.FileLoadException: Could not load file or assembly 'Affdex, Version=3.4.1.1320, Culture=neutral, PublicKeyToken=null' or one of its dependencies. Attempt to load an unverifiable executable with fixups (IAT with more than 2 sections or a TLS section.) (Exception from HRESULT: 0x80131019)File name: 'Affdex, Version=3.4.1.1320, Culture=neutral, PublicKeyToken=null' ---> System.IO.FileLoadException: Attempt to load an unverifiable executable with fixups (IAT with more than 2 sections or a TLS section.) (Exception from HRESULT: 0x80131019) at System.Reflection.RuntimeAssembly.nLoadImage(Byte[] rawAssembly, Byte[] rawSymbolStore, Evidence evidence, StackCrawlMark& stackMark, Boolean fIntrospection, SecurityContextSource securityContextSource) at System.Reflection.Assembly.Load(Byte[] rawAssembly) at Microsoft.Azure.WebJobs.Script.Description.FunctionMetadataResolver.ResolveAssembly(String assemblyName) in C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Description\DotNet\FunctionMetadataResolver.cs:line 232 at Microsoft.Azure.WebJobs.Script.Description.FunctionAssemblyLoadContext.ResolveAssembly(String name) in C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Description\DotNet\FunctionAssemblyLoadContext.cs:line 61 at Microsoft.Azure.WebJobs.Script.Description.FunctionAssemblyLoader.ResolveAssembly(Object sender, ResolveEventArgs args) in C:\projects\azure-webjobs-sdk-script\src\WebJobs.Script\Description\DotNet\FunctionAssemblyLoader.cs:line 60'2017-12-10T12:56:29.574 Unable to find assembly 'Affdex, Version=3.4.1.1320, Culture=neutral, PublicKeyToken=null'. Are you missing a private assembly file?2017-12-10T12:56:29.635 Exception while executing function: Functions.HttpTriggerCSharp1. mscorlib: Exception has been thrown by the target of an invocation. f-HttpTriggerCSharp1__-1535649923: Could not load file or assembly 'Affdex, Version=3.4.1.1320, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.2017-12-10T12:56:29.729 Function completed (Failure, Id=22fce544-1d7f-4262-8c59-dcb962ac0bd1, Duration=211ms)

AffdexWrapper is using .NET library Affdex.dll which is in turn using affdex-native.dll and opencv_ffmpeg248.dll. All for libraries are in the same folder in the root of the app. Affdex documentation mentions that it requires Visual C++ Redistributable runtime for VS 2013

Are there any solutions at this point of time?

Dovydas

[patricklee2]

What is the bitness of the assembly that you are trying to load, 32-bit or 64-bit?
For native assemblies the bitness has to match. The default bitness for azure functions is 32.

[dovisio]

referencing 32 bit libraries and function is 32 bit too. Also I have to mention that function is running from .csx file

[patricklee2]

Can you provide steps to reproduce this error?

[dovisio]

I have pushed Azure Function project to below repository:
https://github.com/dovisio/AffdexFunctionApp32bit

This project works fine locally but fails to load when deployed to Azure.
Thanks in advance

[patricklee2]

I was unable to run the project locally. affdex-native.dll and opencv_ffmpeg248.dll could not be added as references in the project and I was unable to load Affdex.dll. What were the steps you took to install these dlls?

[patricklee2]

Can you try running through the steps at https://blogs.msdn.microsoft.com/benjaminperkins/2017/04/13/how-to-add-assembly-references-to-an-azure-function-app/

[dovisio]

affdex-native.dll and opencv_ffmpeg248.dll cannot be added as references because they both are native libraries.

Affdex.dll works as (I suspect) a CLI/C++ bridge for .NET application.

Affdex.dll is already referenced and project should work locally, but fail when deployed to Azure functions. What errors are you getting? This link might help you https://knowledge.affectiva.com/v3.2/docs/getting-started-with-the-emotion-sdk-for-windows

[dovisio]

Can you try running through the steps at https://blogs.msdn.microsoft.com/benjaminperkins/2017/04/13/how-to-add-assembly-references-to-an-azure-function-app/

This suggestion does not make sense. I have no problem adding managed dependencies.

I'm suspecting that VC++ Redistributable packages are not loading somewhere in the dependency chain. Azure functions environment being a restricted sandbox, I cannot check fusion logs or install missing dependencies etc.

Related issue #Azure/azure-functions-host#1469

[dejanberic]

I am using a nuget library which uses CLI/C++ dll which could not load in Azure Functions but loads correctly on my machine. This dll which needs to be loaded is used to load native dll and run some processes.
Will this kind of thing ever be supported?
If not, what do you suggest I use instead of Azure Functions?
Currently I am using Classic Cloud Services which starts full blown Windows 10 VM.

[jeffhollan]

Updating this issue in that I believe a number of improvements have been made here with Functions v2. It would be worth trying with latest v2 Functions (.NET Core). Not supported for functions v1

[VividDigital]

Is there really no support for native libraries? I'm trying to use the Skiasharp library which works perfectly fine locally but when I deploy to Azure it can't find the assembly. See my issue created here.

I'm using app service too so I don't understand why it can't load/find the assembly. Am I really going to have to scrap functions entirely over this one library?

[Gillibald]

If your native library requires a dynamically linked library by itself it will most likely fail to load in an environment that doesn't let you install these pre-requirements. I suggest building your native library with statically linked dependencies. If that isn't possible I don't think there is an easy way to get around this issue.

[jeffhollan]

Yes native dependencies are definitely supported but need to make sure the library includes them as well as a valid “runtimes” folder and metadata so the .NET host can load in the right native dependencies for the platform

[VividDigital]

I've managed to get this working and it seems the issue was what I initially noticed was different in the issue I referenced in my first post.

When the app is published via VS2019, the runtimes folder gets published to wwwroot. Locally it gets published INSIDE the bin folder. Locally it works and on the server it doesn't.

Uploading the runtimes folder into the bin via ftp wouldn't work because access issues. Same goes with kudu, so I left it at that. Then today I realized I might be able to do it if I don't publish from package(zip). I created a new project and published the other way and that allows writing to the bin so I uploaded the runtimes folder into the bin and it works now.

What is the reason the publishing works differently when deploying? Is this a bug or do I need to do something on my end to make sure it gets published correctly within the bin? I'd like to run from package and not have to manually upload the runtimes folder each time I deploy.

[AlexNosk]

I resolved the same issue by adding libSkiaSharp.dll for win-x86 into the project and specifying “Copy to Output Directory” option in its properties to “Copy Always” value. In this case libSkiaSharp.dll is placed next to SkiaSharp.dll and it works properly. Hope this helps.

[jeffhollan]

@fabiocav curious to get your thoughts here. There's been a few libraries we've run into that aren't getting the bin's correctly copied into the internal bin folder. Is this a thing the functions SDK should be doing, or potentially an issue with how the libraries are communicating dependencies?

[fabiocav]

The comment above leads me to believe this is a defect with the publishing logic and something we need to look at.

@VividDigital do you have a project sample with a repro we can look at? Want to make sure I'm working with your exact setup to rule out any differences in the project configuration.

[VividDigital]

I created a basic function that replicates the issue. Publishes runtimes folder inside bin locally but outside when deployed.
FunctionRuntimesExample.zip

[alaatm]

Having the same issue with SkiaSharp running in v3-preview function. Even though the /runtimes folder, which includes the native libraries, is published next to the /bin folder yet I am still getting Unable to load DLL 'libSkiaSharp' or one of its dependencies: The specified module could not be found. (0x8007007E)

[alaatm]

Adding the following to function project file seem to solve the problem:

<Target Name="PostPublish" AfterTargets="AfterPublish">
    <Exec Command="move $(PublishDir)\runtimes $(PublishDir)\bin" Condition=" '$(OS)' == 'Windows_NT' " />
    <Exec Command="mv $(PublishDir)\runtimes $(PublishDir)\bin" Condition=" '$(OS)' != 'Windows_NT' " />
</Target>

[mattleibow]

I just want to add my 2c here. I think this is not really a question, but a bug or a missing feature. Unless, the documentation needs to me more explicit that a Function App does not work the same way as the rest of .NET Core.

The reason I say this is that with all things .NET Core and NuGet, if there is a native binary in the runtimes folder, then at runtime it is loaded so that the managed app can use (via p/invoke for example) that native binary. This works with console apps, websites, and pretty much everything else. If this is not supported, then why even copy the runtimes folder in the first place?

I hope we get native native support. 😉

Copying from @alaatm, I used MSBuild things to copy files:

<Target Name="CopyRequiredNativeAssets" AfterTargets="Publish">
  <ItemGroup>
    <NativeAssetToCopy Include="$(PublishDir)runtimes\win-x86\native\libSkiaSharp.dll" />
  </ItemGroup>
  <Copy SourceFiles="@(NativeAssetToCopy)" DestinationFolder="$(PublishDir)bin" />
</Target>

I noticed was that the runtime is 32-bit? Seems to be 64-bit hardware but 32-bit OS.

[mattleibow]

@fabiocav I have a test repo here with the extra target to get it to work: https://github.com/mattleibow/SkiaSharpFunctions You should just be able to delete that snippet and see the error appear.

[BHuber-PlanB]

Would be great to have some updates on the problem soon because we switched in our project from Azure App Service backend to Function App backend and now we are facing the same issue with SkiaSharp (we e.g. use it for image downscaling etc.). So now we now have the problem we can't use SkiaSharp with Azure Functions on production environment. We don't get it to work. Even the post of @mattleibow did not work in our case (Azure Function v2 Runtime & .NET Core 2.2).

So for now the only way for us is to look for an alternative to SkiaSharp that works with Azure Function v2 Runtime & .NET Core 2.2 or switching in our XAMARIN Native application back to our Azure App Service backend.

[mattleibow]

@BernhardHuberPlanBGmbH do you have a Linux or a windows function? If you are using a Linux function, you may need to include https://www.nuget.org/packages/SkiaSharp.NativeAssets.Linux/ and copy libSkiaSharp.so instead.

[BHuber-PlanB]

@mattleibow Thanks for the quick reply! We are using a windows function.

[mattleibow]

And manually copying the libSkiaSharp.dll doesn't work? It is working in this example: https://github.com/mattleibow/SkiaSharpFunctions

https://skiasharpfunctions.azurewebsites.net/api/images?text=SkiaSharp%20Functions

[BHuber-PlanB]

@mattleibow When manually copying the libSkiaSharp.dll to .../bin i receive the following message: Could not evaluate 'libSkiaSharp.dll' for extension metadata. Exception message: Bad IL format.

[mattleibow]

That is just a warning, so you can ignore that. The function tasks are trying to process it, but they shouldn't. Probably a bug or we need some way to ignore that. I'll have a look and see maybe there is something.

[BHuber-PlanB]

@mattleibow I manually copied libSkiaSharp.dll to bin directory. After that I tried to publish the Azure Function to Azure via Visual Studio publish functionality. Then the error message 'Could not evaluate 'libSkiaSharp.dll' for extension metadata. Exception message: Bad IL format.' occurred. I reviewed the files of the Azure Function via App Service Editor. libSkiaSharp.dll wasn't uploaded to Azure.

[mattleibow]

@BernhardHuberPlanBGmbH Hmm. Maybe the VS publish option doesn't like this. I'll see if I can adjust some things in the target to only copy AFTER the processing has been done.

[mattleibow]

I had a look and I was still able to publish, even with that error message. But, just in case there are other issues, I'll still have a look and see if we can avoid that error altogether.

[mattleibow]

@BernhardHuberPlanBGmbH I just had to move the copy tasks to after the gen steps (_FunctionsPostPublish):

<Target Name="CopyRequiredNativeAssets" AfterTargets="_FunctionsPostPublish">
  <ItemGroup>
    <NativeAssetToCopy Include="$(PublishDir)runtimes\win-x86\native\libSkiaSharp.dll" />
  </ItemGroup>
  <Copy SourceFiles="@(NativeAssetToCopy)" DestinationFolder="$(PublishDir)bin" />
</Target>

[BHuber-PlanB]

@mattleibow I think one thing that differs in my case is that our Azure Function assembly references another assembly which uses the SkiaSharp library and not the Azure Function assembly itself. So this could maybe the reason why this doesn't work for me.

[mattleibow]

@BernhardHuberPlanBGmbH Ah, that might be it. But, then you can just add the SkiaSharp NuGet to your function project. This way you can do this. Although, it is a bit weird why the runtimes is not copied anyway.

[BHuber-PlanB]

@mattleibow I added SkiaSharp NuGet in both Azure Function itself and the referenced assembly (for testing the copying of the libSkiaSharp.dll) but it is exactly what happens: libSkiaSharp.dll isn't copied and I get the DLL not found error for libSkiaSharp.dll.

[mattleibow]

@BernhardHuberPlanBGmbH Try pulling the repo and deploying it to a function your side. This works for me: https://github.com/mattleibow/SkiaSharpFunctions
Here it is running: https://skiasharpfunctions.azurewebsites.net/api/images?text=SkiaSharp%20Functions

I tried with both DevOps deployment and a right-click publish. I also checked the zip file in the obj directory and I see it in there as well.

[BHuber-PlanB]

@mattleibow I added your target to csproj, but when I publish via VS 2019 to Azure the libSkiaSharp isn't included in bin in ZIP. Looks like VS publish skips the target step. Here's my csproj file:

<Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <TargetFramework>netcoreapp2.2</TargetFramework> <AzureFunctionsVersion>v2</AzureFunctionsVersion> </PropertyGroup> <ItemGroup> <...> </ItemGroup> <ItemGroup> <...> </ItemGroup> <ItemGroup> <None Update="host.json"> <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> </None> <None Update="local.settings.json"> <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> </None> </ItemGroup> <Target Name="CopyRequiredNativeAssets" AfterTargets="_FunctionsPostPublish"> <ItemGroup> <NativeAssetToCopy Include="$(PublishDir)runtimes\win-x86\native\libSkiaSharp.dll" /> </ItemGroup> <Copy SourceFiles="@(NativeAssetToCopy)" DestinationFolder="$(PublishDir)bin" /> </Target> </Project>

[BHuber-PlanB]

@mattleibow It works on my side with your sample project as well. Now I'm unsure why it doesn't work with my own project.

[BHuber-PlanB]

@mattleibow I noticed that even removing the build target from csproj of your project still copies libSkiaSharp.dll So I think there's something I'm missing in my project.

[BHuber-PlanB]

@mattleibow I think the reboot of my local machine solved the copying issue in my own project. It worked after the reboot. Now I'll create a pull request to see if it works using Azure DevOps pipeline as well. Thanks a lot for your help!

[sowsan]

We need some official guidelines on linking native DLLs (both 32 and 64 bit versions) with Azure functions. Seeing this as common requirement for the customers who are modernizing their existing workloads.

[mzhukovs]

@mattleibow @BernhardHuberPlanBGmbH

Not sure what you're seeing on your end, but in my Azure Functions project the runtimes folder is in the bin folder... so need to update this line:
<NativeAssetToCopy Include="$(PublishDir)runtimes\win-x86\native\libSkiaSharp.dll" />
to
<NativeAssetToCopy Include="$(PublishDir)bin\runtimes\win-x86\native\libSkiaSharp.dll" />

I stumbled across this trying to solve a related matter, where System.Drawing requires libgdiplus on Linux, but unfortunately, I could not get it to work (however, I did confirm that the files are correctly getting copied with the below). If anyone has solved a similar problem would much appreciate a tip.

  <Target Name="CopyRequiredNativeAssets" AfterTargets="_FunctionsPostPublish">
    <ItemGroup>
      <NativeAssetsToCopy Include="$(PublishDir)bin\runtimes\linux-x64\native\*.*" />
    </ItemGroup>
    <Copy SourceFiles="@(NativeAssetsToCopy)" DestinationFolder="$(PublishDir)bin" />
  </Target>

[catmanjan]

This is still an issue in 2022

[BenjaminMichaelis]

Still running into this issue, I am using a linux Azure Function. This fixed for me:

  <Target Name="CopyFilesAfterPublish" AfterTargets="PostBuildEvent">
    <Copy SourceFiles="$(TargetDir)runtimes\linux-x64\native\libSkiaSharp.so" DestinationFolder="$(PublishDir)\bin\" />
  </Target>

[BenjaminMichaelis]

Still running into this issue, I am using a linux Azure Function. This fixed for me:

  <Target Name="CopyFilesAfterPublish" AfterTargets="PostBuildEvent">
    <Copy SourceFiles="$(TargetDir)runtimes\linux-x64\native\libSkiaSharp.so" DestinationFolder="$(PublishDir)\bin\" />
  </Target>

Update on #622 (comment).

A better fix because my previous solution in the comment linked above fails on an azure pipeline when running a publish command with the --no-build argument since that step doesn't grab onto the file that I was moving after build. So a better fix is this: (note the change of the AfterTargets trigger).

<Target Name="CopyFilesAfterPublish" AfterTargets="AfterPublish">
  <Copy SourceFiles="$(TargetDir)runtimes/linux-x64/native/libSkiaSharp.so" DestinationFolder="$([System.IO.Path]::GetFullPath('$(PublishDir)'))/bin/" />
</Target>

I also use $([System.IO.Path]::GetFullPath('$(PublishDir)')) because if you look at how msbuild looks for this trigger, it uses the same syntax to get the full path instead of the relative path consistently. This has seemed to be the best fix even if using SkiaSharp.NativeAssets.Linux.NoDependencies and works for both deploy directly to an azure function and via an azure pipeline.