Load ONNX model creates temp file

[ntpthinh]

System information

• OS version/distro: Windows
• .NET Version (eg., dotnet --info): .NET framework 4

Issue

• What did you do?
  Load model saved by ML.NET, the base model is in ONNX format
• What happened?
  Everytime I load the model, a new onnx model file is created in %temp% folder and it is not deleted after releasing model object or after exiting the application. File path: %temp%\7bf1a39c-b65c-4874-927b-2bacbbf57b15\model.onnx
  I tried to delete the file when the model was predicting but there is no error or exception and the result is the same.
• What did you expect?
  This file should be deleted or better not be created since it exposes our model to the user consumes too much user's memory.

Source code / logs

Please paste or attach the code or logs or traces that would be helpful to diagnose the issue you are reporting.

[harishsk]

Can you please share the code you are using to load the onnx model?

[ntpthinh]

I simply load the saved model by
ITransformer trainedModel = mlContext.Model.Load(modelLocation, out DataViewSchema modelSchema);

Code for saving model:

            var pipeline = mlContext.Transforms.LoadImages(
                    outputColumnName: "image_object",
                    imageFolder: null,
                    inputColumnName: nameof(ImageData.ImagePath))
            // step 2: resize the images
            .Append(mlContext.Transforms.ResizeImages(
                outputColumnName: "image_object_resized",
                imageWidth: ImageSettings.imageWidth,
                imageHeight: ImageSettings.imageHeight,
                inputColumnName: "image_object"
               ))

            // step 3: extract pixels in a format the TF model can understand
            // these interleave and offset values are identical to the images the model was trained on
            .Append(mlContext.Transforms.ExtractPixels(
                outputColumnName: ModelSettings.inputTensorName,
                inputColumnName: "image_object_resized",
                interleavePixelColors: ImageSettings.channelsLast,
                offsetImage: ImageSettings.mean,
                scaleImage: ImageSettings.scale))
            .Append(mlContext.Transforms.ApplyOnnxModel(modelFile: _modelFilePath, outputColumnName: ModelSettings.outputTensorName, inputColumnName: ModelSettings.inputTensorName));

            ITransformer mlModel = pipeline.Fit(dataView);
            mlContext.Model.Save(mlModel, dataView.Schema, _outputModelFilePath);

[harishsk]

So, you have a .onnx file that you are loading in your pipeline with mlContext.Transforms.ApplyOnnxModel. And then you save the whole pipeline into another ML.NET model.
And you see the temp file that doesn't get deleted while loading that ML.NET model.

Is my understanding correct?

[ntpthinh]

That's right.
Since mlContext.Transforms.ApplyOnnxModel only allows file url but I can only provide stream in my application so I have to work around like that.

[harishsk]

@pthinh14 Thanks for reporting the issue. It will be fixed as part of PR #5348 and be part of the next release.

[antoniovs1029]

As a side note: @pthinh14 , if you're giving your customers the ML.NET model file they'll load with mlContext.Model.Load(modelLocation, out DataViewSchema modelSchema); then please note that the ML.NET model file is simply a .zip file which can be unzipped, and inside there will be a file called OnnxModel that will actually be your onnx model. I think they'll simply need to open it on a notepad editor, and remove a couple of characters at the beginning, and that'll be enough for them to have access to your original onnx model.

When saving your ML.NET model to disk, here is the code where the original onnx model gets saved into the .zip file:

machinelearning/src/Microsoft.ML.OnnxTransformer/OnnxTransform.cs

Line 307 in a769365

ctx.SaveBinaryStream("OnnxModel", w => { w.WriteByteArray(File.ReadAllBytes(Model.ModelFile)); });

I mention this as you've expressed you don't want your onnx model to be exposed to your customers.