Unable to load DLL 'FastTreeNative' in Azure Function

[hejingkan2005]

System information

• OS version/distro: win10 (10.0.19041 Build 19041)
• .NET Version (eg., dotnet --info): .net core 3.1

Issue

• What did you do?
  ML.net 1.5.1
• What happened?
  It is no issue to run sample code in console app, but it's repeatedly reporting below error when run CreateModel() from a testing azure function:
  =============== Training model ===============
  [7/16/2020 7:41:11 AM] Executed 'NormalizeFunction' (Failed, Id=64d8e39f-5a8b-40e9-b335-3cf2cbf4ce45)
  [7/16/2020 7:41:11 AM] System.Private.CoreLib: Exception while executing function: NormalizeFunction. System.Private.CoreLib: One or more errors occurred. (Unable to load DLL 'FastTreeNative' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E)). Microsoft.ML.FastTree: Unable to load DLL 'FastTreeNative' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E).
• What did you expect?
  It should run without exception as it did in console app

Source code / logs

Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.

[michaelgsharp]

Hey @hejingkan2005, it should work just fine. When you deployed to azure functions did you use use these steps? https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/serve-model-serverless-azure-functions-ml-net. In theory it should just involved adding the FastTree nuget so the DLL goes there automatically when you deploy it.

Can you verify that you followed those instructions and have the FastTree nuget and let me know?

[hejingkan2005]

Thank for your reply @michaelgsharp. But the problem here is little bit different. Instead of consuming 'mlmodel.zip', I'm expecting to generate 'mlmodel.zip' in Azure Function by reading training datasets from azure storage (e.g. blob). So the link you pasted does not apply to my case. I'm sure that I've FastTree nuget package installed as dependency.

[mstfbl]

Hi @hejingkan2005 ,

Can you share some code samples that you have with which you've obtained the "unable to load DLL" error? I will try to reproduce your problem.

By the way, we do not recommend training and generating a model in Azure Functions due to the limitations on timeouts in Azure Functions's service, in addition to potential limitations that may arise from hardware specs. Instead we'd recommend you use a VM to generate your model normally while using a dataset that's stored in an Azure storage facility.

In addition, what version of the Azure Functions SDK are you using?

[hejingkan2005]

Hi @mstfbl,
I'm using Azure Functions SDK version 3.0.8, please refer to the .csproj for more detail:

<PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.0.0" />
<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Storage" Version="3.0.10" />
<PackageReference Include="Microsoft.Extensions.ML" Version="1.5.1" />
<PackageReference Include="Microsoft.ML" Version="1.5.1" />
<PackageReference Include="Microsoft.ML.FastTree" Version="1.5.1" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.8" />
<PackageReference Include="System.Linq" Version="4.3.0" />

It looks not timeout issue with azure function as I get quick response when executing lines highlighted below,

[FunctionName("MLModelGenerateFunction")]
public async Task Run([TimerTrigger("*/5 * * * * *")] TimerInfo myTimer, ILogger log)
{
            ConcurrentBag<ModelInput> datasets = new ConcurrentBag<ModelInput>();
            **ModelBuilder.CreateModel();**
}

public static void CreateModel()
{
            // Load Data
            IDataView trainingDataView = mlContext.Data.LoadFromTextFile<ModelInput>(
                                            path: TRAIN_DATA_FILEPATH,
                                            hasHeader: true,
                                            separatorChar: ',',
                                            allowQuoting: true,
                                            allowSparse: false);

            // Build training pipeline
            IEstimator<ITransformer> trainingPipeline = BuildTrainingPipeline(mlContext);

            // Train Model
            **ITransformer mlModel = TrainModel(mlContext, trainingDataView, trainingPipeline);**

            // Evaluate quality of Model
            Evaluate(mlContext, trainingDataView, trainingPipeline);

            // Save model
            SaveModel(mlContext, mlModel, MODEL_FILEPATH, trainingDataView.Schema);
}

Here is the error I got in my azure function console:
=============== Training model ===============
[7/22/2020 8:13:26 AM] Executed 'NormalizeFunction' (Failed, Id=c630947c-9b0a-4bf0-8c9f-8c133bd4c42b)
[7/22/2020 8:13:26 AM] System.Private.CoreLib: Exception while executing function: NormalizeFunction. System.Private.CoreLib: One or more errors occurred. (Unable to load DLL 'FastTreeNative' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E)). Microsoft.ML.FastTree: Unable to load DLL 'FastTreeNative' or one of its dependencies: The specified module could not be found. (Exception from HRESULT: 0x8007007E).
[7/22/2020 8:13:30 AM] Executing 'NormalizeFunction' (Reason='Timer fired at 2020-07-22T16:13:30.0051020+08:00', Id=4a63060a-b53b-4c76-a24b-73fc8549424a)

I run the same code in console app, there is no problem at all.