[WIP: Feb2023] Pytorch to TorchSharp using .NET interactive (.NET c# kernel)

[GeorgeS2019]

After ML.NET, PyToCs is the second most Stars C# github repos that is using TorchSharp!
Thanks for @toolgood

============ 2023 ===========================================
Recently @toolgood contributed the additional PyToCS (Python To CSharp) codes needed to move these discussions forwards.

============ 2022 ===========================================
This will be a joint collaboration between the TorchSharp, :NET Interactive teams, and perhaps others

Use Case

I would like to open a working PyTorch notebook using .NET interactive to determine which of these PyTorch statements in python NOT supported in TorchSharp.

For each of these "nonsupported" PyTorch statements, add an issue to TorchSharp GitHub

Please note

These notebooks will be those used in PyTorch courses. More on PyTorch syntax and NOT a mix with other python libraries like e.g. numpy

[GeorgeS2019]

• torch.numel() exists but not torch.numel( tensor)
• torch.InitTensor e.g. torch.tensor([5, 3]).type(torch.IntTensor);
• torch.full no unit test
• torch.new_full no unit test

[GeorgeS2019]

@NiklasGustafsson

Can not find

new_full. torch.full unit tests, please share links

[NiklasGustafsson]

Line 1549 in TestTorchTensor.cs

[GeorgeS2019]

Found torch.full

[GeorgeS2019]

Not new_full

full_like => yes

[NiklasGustafsson]

Right, that's what I said. I'm adding one for new_full()

[GeorgeS2019]

numpy_arr = np.array([[1.0, 2.0, 3.0], 
                      [10.0, 20.0, 30.0],
                      [100.0, 200.0, 300.0]])
tensor_from_numpy = torch.from_numpy(numpy_arr)

How to handle that in TorchSharp?

var tensor_from_numpy = torch.from_array();

[GeorgeS2019]

using NumpyDotnet;

var a1 = np.asarray(new float[]{1.0f, 2.0f, 3.0f});
var a2 = np.asarray(new float[]{110.0f, 20.0f, 30.0f});
var a3 = np.asarray(new float[]{100.0f, 200.0f, 300.0f});

var numpy_arr = new ndarray[]{ a1,a2, a3};

[NiklasGustafsson]

We have no intent to integrate TorchSharp and Numpy.NET. torch.from_array() takes N-D .NET arrays.

[GeorgeS2019]

var numpy_arr1 = new float[3,3]{ {1.0f, 2.0f, 3.0f},
                                 {10.0f, 20.0f, 30.0f},
                                 {100.0f, 200.0f, 300.0f}};
var tensor_from_numpy = torch.from_array(numpy_arr1);

[GeorgeS2019]

PyTorch to TorchSharp: the python way

PyTorch

my_tensor = torch. Tensor([[1,2,3],[4,5,6]], dtype = torch.float32)

PyToCs

using System.Collections.Generic;
public static class testModule {
    
    public object my_tensor = torch.tensor(new List<object> {
        new List<object> {
            1,
            2,
            3
        },
        new List<object> {
            4,
            5,
            6
        }
    }, dtype: torch.float32);
}

Desired c# TorchSharp's python way

var my_tensor = torch. Tensor(new int[,]{{1,2,3},{3,5,6}}, ScalarType.Float32)

In PyToCs terms

• FieldName: my_tensor
• Method: torch.tesnsor()
• Arguments:
  • [[1,2,3], [4,5,6]]
  • dtype = torch.float32

suggestion

Once torch is detected in method, TorchSharp specific argument translation is applied

• [[1,2,3], [4,5,6]] => new int[,]{{1,2,3},{3,5,6}}
• dtype: torch.float32 => ScalarType.Float32

I need idea which part of the PyToCs codes for implementing the TorchSharp specific statement translation

[GeorgeS2019]

FYI

@NiklasGustafsson
@kaiidams
With this, we can scan PyTorch notebooks and extract the missing TorchSharp binding APIs

[NiklasGustafsson]

@GeorgeS2019 -- unfortunately, I can't pretend that I know what it is you are proposing.

What does this 'PyToC' tool do, who would use it, and why/when?

[GeorgeS2019]

PyToCs is python to cSharp.

It is WIP.

I am getting the creator of PyToCs to help.
Also, put it here for others interested to contribute.

Currently, based on my ongoing feasibility analysis, it should be possible to create a .NET program (user contribution) to load PyTorch Notebooks available in Github and extract PyTorch statements with "Torch" to identify PyTorch APIs that are still missing in TorchSharp

This could serve a few purposes

• To track the missing PyTorch APIs (typically used in real world but still missing) in TorchSharp. This could speed up TorchSharp towards eventually version 0.98 to 0.99
• The report also guides others interested in TorchSharp to focus on the missing APIs
• Eventually, the PyToCs with Torchsharp additions, could help beginners to port PyTorch code to TorchSharp, knowing where to look for things to make the fine adjustments.

[kaiidams]

You usually need more manual fixes after converting one language to another by converters. Often wild Jupyter notebooks use any dependencies available and some Python grammar is hard to convert without generating lines that are difficult to understand for humans.

I ported a couple of models from PyTorch to TorchSharp and I see that support of torch operators are good enough. Sometimes they needs works because of diversion from PyTorch. PyTorch C++ and Python operators are generated from a single YAML file https://github.com/pytorch/pytorch/blob/ccac8d13d5988de7302551a5df460072eb918683/aten/src/ATen/native/native_functions.yaml . C# should be aligned as well.

I think we miss many I/O and preprocessing, which are hard to write end-to-end samples. We usually have old GitHub issues already. I hope @GeorgeS2019 can prioritize them. I think they often depends on other dotnet teams and are in unknown states, not receiving replies.

[GeorgeS2019]

@NiklasGustafsson

What do you think of porting e.g. Microsoft python 101 bootcamp class (especially those teaching pytorch) into TorchSharp?

The ported notebooks could serve as the starting education materials for colleges and universities.

The NEXT significant milestones for TorchSharp?
#733
cc @@toolgood

[NiklasGustafsson]

What do you think of porting e.g. Microsoft python 101 bootcamp class (especially those teaching pytorch) into TorchSharp?

I have and continue to strongly encourage the creation of more examples in 'dotnet/TorchSharpExamples,' which is where something like this belongs.

[GeorgeS2019]

@NiklasGustafsson

The Microsoft bootcamp class uses:

• numpy: 2-tensors.ipynb
• matplotlib:
  • 3-data.ipynb
  • 9-quickstart.ipynb
• torch.onnx: 8-inference.ipynb
• torchvision:
  • 3-data.ipynb
  • 4-transforms.ipynb
  • 5-model.ipynb
  • 7-optimization.ipynb
  • 8-inference.ipynb
  • 9-quickstart.ipynb

We need discussion how to address these in .NET

There are TWO options moving forwards
(a) Explore how the new features in PolyGlot could address that.
(b) The non-PolyGlot way is to combine Python.NET and TorchSharp

For option (b), we need to discuss e.g NumpyDotNet. It is a topic that ALL NEW TorchSharp users will ask and without addressing it, this will confuse the new users

Python.NET has ported MatPlotLib for .NET. The alternative is to use the recommended option for PolyGlot

All these are topics that TorchSharp and PolyGlot teams need to coordinate.

We also need to have all the TorchVision methods used in these bootcamp class. Why having the right TorchVision folder structures will help to coordinate these missing methods

TorchVision keep the folder structures consistent between pytorch and Torchsharp?

[NiklasGustafsson]

As far as numpy goes, and as we have discussed elsewhere, there's no need for that. In the bootcamp notebooks, it's only used to create tensors from arrays, and we have that in TorchSharp already -- from_array, etc. I understand the general desire for a solid numerics library for .NET, but that's far outside the scope of TorchSharp, and not required for these tutorials. We don't need a .NET numerics library for n-d arrays, we already have those in .NET.

Matplotlib is another matter, but I would prefer that we'd rely on a .NET solution, instead of requiring Python interop in order to get things to work. In fact, I will be very reluctant to merge a PR that isn't pure .NET, even in TorchSharpExamples. That's the philosophy behind TorchSharp -- no Python required. You may agree or disagree with that stance (there are good arguments both ways), but that's a core principle. There are .NET-based plotting libraries, and we should use them for this.

There's a third library used in the bootcamp -- Pandas. However, the only thing I believe it's used for is to read CSV files, which isn't trivial, but neither is it rocket science. There are several CSV libraries for .NET, too.

[GeorgeS2019]

@NiklasGustafsson

I agree with most of your points.

Just like how TorchSharp has created a POSSIBLE path forwards to do deep learning (pytorch) for .NET

We need to define the guidelines (e.g. wiki) and if possible a more friendly PyTorch-To-Torchsharp utilities or examples for

• numpy
• Matplotlib
• Pandas ( I think ML.NET has relevant DataFrame)

These are WHAT MISSING, a CONSENSUS how to address these and in way (e.g. PolyGlot) that NEW USERS can just take one of these PyToch notebooks and start learning TorchSharp by porting them over using the CONSENSUS guideline.

[NiklasGustafsson]

BTW, all of the bootcamp courses look fine. The CV content is fairly well covered in the example, and some of the NLP ones, but I agree that it would be great to port the specific course material to TorchSharp.

[GeorgeS2019]

We discussed this before, the day when we have college or university lectures start using TorchSharp though PolyGlot, then the TorchSharp adoption will grow much faster then WHAT has been achieved NOW.

This is not just dotnet TorchSharp, ML.NET, and PolyGlot teams, The business of pushing hardware and software to Colleges and Universities and much more.

[NiklasGustafsson]

We discussed this before, the day when we have college or university lectures start using TorchSharp though PolyGlot, then the TorchSharp adoption will grow much faster then WHAT has been achieved NOW.

You make it sound like TorchSharp is somehow in competition with PyTorch and .NET with Python. That is not why we built this library, or ML.NET for that matter. We are not trying to convert any Python developers to use .NET. We want to give existing .NET developers the tools to do machine learning and neural networks without having to leave the .NET ecosystem. That's it.

[GeorgeS2019]

TorchSharp is somehow in competition with PyTorch and .NET with Python

No competition, But more of keeping the project alive and growing so I could stay doing deep learning in .NET.

FYI: Very concern of the future of .NET for Spark. It seems no one is pushing from dotnet side. The next step on TorchSharp for me is to combine TorchSharp and .NET for Spark, in same way as PyTorch and Spark (but in .NET instead of python)

[NiklasGustafsson]

Sure, we can and should come up with ideas on how to accomplish things. As far as a consensus goes, I don't think we need that. TorchSharp is, and will remain, a thin layer on top of libtorch. That's it. How folks want to plot data or integrate with other libraries is not something TorchSharp should mandate or have a strong opinion on.

Libraries and repos that take a dependency on TorchSharp are free to do whatever they want and involve the runtime stacks that make sense for their users.

For TorchSharp, rule #1 is that it must be a .NET-based solution. We have nothing against libraries and solutions that involve Python in the runtime stack -- it's an excellent solution for many scenarios. That said, TorchSharp's #1 architectural principle is that it does not depend on Python at runtime in any way. Library code going in dotnet/TorchSharp, and examples going in dotnet/TorchSharpExamples should be .NET-based.