[WIP: Feb2023] Implement Torch.ONNX

[GeorgeS2019]

[Sep2022]

Long-term feature request

Motivation

There has been some recent effort in C and C++ API in OnnxRuntime (ORT) to make these feature requests feasible

• Deprecate custom op api

There is an increasingly more interest among.NET community to integrate contributed operators

• C# pre and postprocessing with opencvsharp for onnxruntime
• [Documentation]: Overview of C# Workflow for consuming "Augmented" Onnx model with Custom Operators
• [Feature Request: c#] Provide Onnxruntime related Nugets consisting of contributed Operators as shared library

There has been more effort in improving the documentation on integrating/registrating custom/contributed operators

• [Documentation] How to register custom operator in onnxruntime
• [Issue labeler] Separate out C# api as separate label

Impact on Deep AI: NLP/Computer Vision experience using ML.NET

• Contrib Operator Schemas list Deep NLP/Computer vision contributed operators curated by Onnx industry partners

use case 1

• Evaluate the feasibility to use TorchSharp to integrate custom operators and export to ONNX.
• Use ML.NET to register the integrated custom operators used in the exported ONNX model before consuming the ONNX model

use case 2

• Instead of integrating custom operators as stated in use case 1, integrate existing contributed operators distributed by ONNXRUNTIME
• Use ML.NET to register the integrated contributed operators used in the exported ONNX model before consuming the ONNX model

Mid-term feature request

• Implement Torch.Onnx to export Onnx from TorchSharp

[GeorgeS2019]

[Feb2022]

Torch.ONNX

A recent model.load issue suggests how the torchsharp community could coordinate to eventually make loading ONNX feasible in torchsharp.

Any comment and suggestion how relevant is this feature for torchsharp community.

==========================================================================

This discussion could impact Feedback to the plan for Torch integration to ML.NET

Will TorchSharp.ONNX feature redundant compared to ML.NET load ONNX feature for the .NET community?

@michaelgsharp => Currently this integration is being planned

I am posting here to gather more feedback for the integration team from TorchSharp to ML.NET

[GeorgeS2019]

Discussion on implementing TorchSharp convert TorchScript to ONNX

#379

#644 (comment)

[alex4998]

Not sure where it is better to leave a comment, here or in the other thread. This thread topic definitely fits better. I'm trying to find ends about exporting models trained with TorchSharp to ONNX. There are a few mentionings out there that the work on it is in progress and most of them dated back to around mid-2022. Does anyone know the status on TorchSharp export to ONNX? My question is partially related to ML.NET and presumably its ability to load and run ONNX model even now. Also, as I understand there is another in progress work to bring TorchSharp functionality to ML.NET which again looks like incomplete yet. What I was thinking about doing in meanwhile is using TorchSharp to train my model, save/export it as ONNX, then load and run it in ML.NET. Maybe somebody can suggest how to do it. So, far I could train a model with TorchSharp and save it, but it looks like that file format is very specific to TorchSharp/PyTorch/torch. How do I use it in ML.NET then?

[GeorgeS2019]

One of the main motivations of TorchSharp when @NiklasGustafsson actively took over has always been to keep DEEP machine learning activities within the .NET.

This motivation is no longer doubt, but needs further optimization, including, eventually and hopefully, adding more PyTorch features e.g. support of ONNX and eventually custom operators

KEEP REQUESTING! The Microsoft Team decides based on how strong is a particular request.

[NiklasGustafsson]

Export to ONNX is not currently a priority for the TorchSharp team at Microsoft (i.e. me :-)), as you can tell from the lack of activity on that front. At the moment, we are studying how (and whether) we can accommodate PyTorch 2.0.

Adding @luisquintanilla for the ML.NET perspective.

Contributions are, however, always welcome.

[GeorgeS2019]

Import pytorch exported onnx model using TorchSharp

using ChatGPT

To use torch.onnx in TorchSharp, you need to perform the following steps:

1. Train your PyTorch model in Python.
2. Export the model to the ONNX format using torch.onnx.
3. Load the ONNX model in TorchSharp.

Here's a sample code in Python that trains a simple neural network and exports it to the ONNX format:

import torch
import torch.nn as nn

# Define the neural network
class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(4, 10)
        self.fc2 = nn.Linear(10, 3)
    
    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

# Create an instance of the neural network
model = Net()

# Train the model on dummy data
dummy_input = torch.randn(1, 4)
torch.onnx.export(model, dummy_input, "model.onnx")

Once you have exported the model to the ONNX format, you can load it in TorchSharp using the following code:

using Torch;
using System;

namespace LoadONNXModel
{
    class Program
    {
        static void Main(string[] args)
        {
            // Load the ONNX model
            var model = TorchModel.Load("model.onnx");
            
            // Run a sample input through the model
            var input = new Tensor(new[] { 1f, 2f, 3f, 4f }, new[] { 1, 4 });
            var output = model.Forward(input);
            
            // Display the output
            Console.WriteLine(output);
        }
    }
}

This code loads the model.onnx file, creates a TorchModel object, and runs a sample input through the model to produce an output.

[NiklasGustafsson]

This is an example of ChatGPT not getting it right.

1. It should be 'using TorchSharp'
2. TorchModel is not a class in TorchSharp, and if it were, it woudn't be in the top namespace.
3. TorchSharp doesn't support loading ONNX models, but there is a .NET API to do so -- you don't need TorchSharp to load a trained ONNX model.
4. The method to invoke the logic of a TorchSharp module is 'forward', not 'Forward'
5. Console.WriteLine would use the default version of 'ToString(),' which will not print out something very useful.

It appears like ChatGPT is doing what it thinks is right in Python and then translating it to C#. I'm not surprised, since there is precious little TorchSharp code out there for it to be trained on.