Bringing TorchVision models into C++ API

[ShahriarRezghi]

🚀 Feature

Hi. I wanted to bring TorchVision models into C++ API.

Motivation

I was working on a project with the C++ front-end and I wanted to test networks like AlexNet and VGG and ResNet and inception to see the difference in accuracy. I searched a bit and I found out that I must make a Torch script module from a TorchVision model but the problem is that I want to train with the C++ front-end and script modules are either in train or eval mode and this cannot be changed after exporting.

Pitch

I want to implement and bring these models into the C++ API.

Alternatives

One of the alternatives can be exporting a model in both train and eval mode and training the train module and somehow copy the weights of them to eval module and get predictions from there.

Additional context

I read the source code of TorchVision and was able to implement and test AlexNet and VGG and implement ResNet.

[ezyang]

This seems reasonable, though I'd say this would probably live in pytorch/vision in that case https://github.com/pytorch/vision (or maybe a separate repo? Not sure.) cc @fmassa @soumith

[ShahriarRezghi]

Hi @ezyang
I'm sorry but it has been 17 days since I opened this suggestion and there has been no progress so far. I don't know if it's OK to remind you of this issue or not so I'm sorry in advance.

[soumith]

As Edward mentioned, it's a pretty good suggestion, but we should put them in https://github.com/pytorch/vision and not in https://github.com/pytorch/pytorch

Go ahead and start working on it, and thank you :)

[fmassa]

Yes, this is a good thing to have!

One thing that would be worth keeping in mind is how to keep both the C++ and the Python version of the models in sync.

[ShahriarRezghi]

Hi @soumith
I will fork https://github.com/pytorch/vision and put the code inside vision/torchvision/cmodels and put the cmake file inside vision//torchvision and run cmake from vision/setup.py.
Is this the correct way to do this?

[ShahriarRezghi]

@fmassa
By keeping them in sync you mean updating c++ side when python side changes or them being compatible like one reading the saved model file of another?

[fmassa]

@ShahriarSS I meant when Python code changes, it would be good to have a way of checking that the C++ code should change as well

[ShahriarRezghi]

@fmassa Sorry I misunderstood. I am trying to write the code that matches the python code so it will be easy to change the code when the python code changes.
And also I pressed the close button accidentally when commenting this message