Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add conversion from ONNX to PyTorch to allow cross conversion TensorFlow->ONNX->PyTorch #133

Open
valeriosofi opened this issue Jan 2, 2023 · 9 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed speedster Issue related to the Speedster App

Comments

@valeriosofi
Copy link
Collaborator

At the moment we support only PyTorch to ONNX and TensorFlow to ONNX conversions. We could test and use this repo to convert an ONNX model to PyTorch in order to support TensorFlow to PyTorch conversion.

@valeriosofi valeriosofi added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers labels Jan 2, 2023
@SuperSecureHuman
Copy link
Contributor

This also can be used to implement onnx to pytorch conversion
(https://github.com/nebuly-ai/nebullvm/blob/main/nebullvm/operations/conversions/converters.py#L126)

Or is it implemented elsewhere?

@valeriosofi
Copy link
Collaborator Author

Hi @SuperSecureHuman, yep as you can see the pytorch_conversion method is not implemented for now, but we should test the repository linked above and if it works well we could use it to implement the method. If you want to contribute feel free to assign yourself the issue :)

@SuperSecureHuman
Copy link
Contributor

SuperSecureHuman commented Jan 3, 2023

I've started working (by directly importing the onnx2torch module). Not sure if I can assign myself to this because its gonna take me some time to understand the convertors api (and overall layout of all other apis).

Edit: I also have 0 experience with onnx 😅

@valeriosofi
Copy link
Collaborator Author

Thanks, your help is very appreciated! Of course take your time to understand the code, and if you have any questions about how it works, just ask ;)

SuperSecureHuman added a commit to SuperSecureHuman/nebullvm that referenced this issue Jan 3, 2023
@SuperSecureHuman
Copy link
Contributor

from nebullvm.operations.conversions.onnx import convert_onnx_to_torch
import onnx
onnx_model_path = '/home/venom/Downloads/mobilenetv2-12.onnx'
onnx_model = onnx.load(onnx_model_path)
output_file_path = '/home/venom/Downloads/model.pt'
device = 'cpu'
convert_onnx_to_torch(onnx_model, output_file_path, device)

For now the convertion works without error

image

Will try working further to improve it

Meanwhile, it would be great, if you could edit parts of my commit to make it more "module like" :)

@valeriosofi
Copy link
Collaborator Author

Great! Maybe check also with another couple of models to see if they are converted without errors as well. After that you could implement also the method pytorch_conversion in the TensorflowConverter class, where the model should be first converted to ONNX and then to PyTorch :)

@diegofiori diegofiori added the speedster Issue related to the Speedster App label Jan 9, 2023
@SuperSecureHuman
Copy link
Contributor

Update: Sorry for the delay

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' 
from nebullvm.operations.conversions.onnx import convert_onnx_to_torch
import onnx
onnx_model_path = '/home/venom/Downloads/mobilenetv2-7.onnx'
onnx_model = onnx.load(onnx_model_path)
output_file_path = '/home/venom/Downloads/model.pt'
device = 'cpu'
outfile = convert_onnx_to_torch(onnx_model, output_file_path, device)
if outfile is not None:
    print("Converted successfully")
    print(outfile)
else:
    print("Conversion failed")

Now, added error handling

image

Commit - cb74ee8

@SuperSecureHuman
Copy link
Contributor

onnx2torch module, has the requirement of opset version 9 (minimum)

In the above example, the model used opset7 (hence failed)

Trying to convert all the models in https://github.com/onnx/models

Open To view all the models from ONNX repo
Model Status
bertsquad-10.onnx Conversion failed
bertsquad-12-int8.onnx Conversion failed
bertsquad-12.onnx Conversion failed
bertsquad-8.onnx Conversion failed
bidaf-9.onnx Conversion failed
gpt2-10.onnx Conversion failed
gpt2-lm-head-10.onnx Conversion failed
gpt2-lm-head-bs-12.onnx Conversion failed
roberta-base-11.onnx Converted successfully
roberta-sequence-classification-9.onnx Conversion failed
t5-decoder-with-lm-head-12.onnx Converted successfully
t5-encoder-12.onnx Converted successfully
age_googlenet.onnx Converted successfully
gender_googlenet.onnx Converted successfully
arcfaceresnet100-8.onnx Conversion failed
emotion-ferplus-2.onnx Conversion failed
emotion-ferplus-7.onnx Conversion failed
emotion-ferplus-8.onnx Conversion failed
version-RFB-320.onnx Conversion failed
version-RFB-640.onnx Conversion failed
bvlcalexnet-12-int8.onnx Conversion failed
bvlcalexnet-12-qdq.onnx Conversion failed
bvlcalexnet-12.onnx Conversion failed
bvlcalexnet-3.onnx Conversion failed
bvlcalexnet-6.onnx Conversion failed
bvlcalexnet-7.onnx Conversion failed
bvlcalexnet-8.onnx Conversion failed
bvlcalexnet-9.onnx Conversion failed
caffenet-12-int8.onnx Conversion failed
caffenet-12-qdq.onnx Conversion failed
caffenet-12.onnx Conversion failed
caffenet-3.onnx Conversion failed
caffenet-6.onnx Conversion failed
caffenet-7.onnx Conversion failed
caffenet-8.onnx Conversion failed
caffenet-9.onnx Conversion failed
densenet-12-int8.onnx Conversion failed
densenet-12.onnx Converted successfully
densenet-3.onnx Conversion failed
densenet-6.onnx Converted successfully
densenet-7.onnx Conversion failed
densenet-8.onnx Conversion failed
densenet-9.onnx Converted successfully
efficientnet-lite4-11-int8.onnx Conversion failed
efficientnet-lite4-11-qdq.onnx Conversion failed
efficientnet-lite4-11.onnx Converted successfully
googlenet-12-int8.onnx Conversion failed
googlenet-12-qdq.onnx Conversion failed
googlenet-12.onnx Conversion failed
googlenet-3.onnx Conversion failed
googlenet-6.onnx Conversion failed
googlenet-7.onnx Conversion failed
googlenet-8.onnx Conversion failed
googlenet-9.onnx Conversion failed
inception-v1-12-int8.onnx Conversion failed
inception-v1-12-qdq.onnx Conversion failed
inception-v1-12.onnx Conversion failed
inception-v1-3.onnx Conversion failed
inception-v1-6.onnx Conversion failed
inception-v1-7.onnx Conversion failed
inception-v1-8.onnx Conversion failed
inception-v1-9.onnx Conversion failed
inception-v2-3.onnx Conversion failed
inception-v2-6.onnx Conversion failed
inception-v2-7.onnx Conversion failed
inception-v2-8.onnx Conversion failed
inception-v2-9.onnx Converted successfully
mnist-1.onnx Conversion failed
mnist-12-int8.onnx Conversion failed
mnist-12.onnx Conversion failed
mnist-7.onnx Conversion failed
mnist-8.onnx Conversion failed
mobilenetv2-10.onnx Converted successfully
mobilenetv2-12-int8.onnx Conversion failed
mobilenetv2-12-qdq.onnx Conversion failed
mobilenetv2-12.onnx Converted successfully
mobilenetv2-7.onnx Conversion failed
rcnn-ilsvrc13-3.onnx Conversion failed
rcnn-ilsvrc13-6.onnx Conversion failed
rcnn-ilsvrc13-7.onnx Conversion failed
rcnn-ilsvrc13-8.onnx Conversion failed
rcnn-ilsvrc13-9.onnx Conversion failed
resnet101-v1-7.onnx Conversion failed
resnet101-v2-7.onnx Conversion failed
resnet152-v1-7.onnx Conversion failed
resnet152-v2-7.onnx Conversion failed
resnet18-v1-7.onnx Conversion failed
resnet18-v2-7.onnx Conversion failed
resnet34-v1-7.onnx Conversion failed
resnet34-v2-7.onnx Conversion failed
resnet50-caffe2-v1-3.onnx Conversion failed
resnet50-caffe2-v1-6.onnx Conversion failed
resnet50-caffe2-v1-7.onnx Conversion failed
resnet50-caffe2-v1-8.onnx Conversion failed
resnet50-caffe2-v1-9.onnx Converted successfully
resnet50-v1-12-int8.onnx Conversion failed
resnet50-v1-12-qdq.onnx Conversion failed
resnet50-v1-12.onnx Converted successfully
resnet50-v1-7.onnx Conversion failed
resnet50-v2-7.onnx Conversion failed
shufflenet-3.onnx Conversion failed
shufflenet-6.onnx Conversion failed
shufflenet-7.onnx Conversion failed
shufflenet-8.onnx Conversion failed
shufflenet-9.onnx Converted successfully
shufflenet-v2-10.onnx Conversion failed
shufflenet-v2-12-int8.onnx Conversion failed
shufflenet-v2-12-qdq.onnx Conversion failed
shufflenet-v2-12.onnx Conversion failed
squeezenet1.0-12-int8.onnx Conversion failed
squeezenet1.0-12.onnx Conversion failed
squeezenet1.0-13-qdq.onnx Conversion failed
squeezenet1.0-3.onnx Conversion failed
squeezenet1.0-6.onnx Conversion failed
squeezenet1.0-7.onnx Conversion failed
squeezenet1.0-8.onnx Conversion failed
squeezenet1.0-9.onnx Conversion failed
squeezenet1.1-7.onnx Conversion failed
vgg16-12-int8.onnx Conversion failed
vgg16-12.onnx Converted successfully
vgg16-7.onnx Conversion failed
vgg16-bn-7.onnx Conversion failed
vgg19-7.onnx Conversion failed
vgg19-bn-7.onnx Conversion failed
vgg19-caffe2-3.onnx Conversion failed
vgg19-caffe2-6.onnx Conversion failed
vgg19-caffe2-7.onnx Conversion failed
vgg19-caffe2-8.onnx Conversion failed
vgg19-caffe2-9.onnx Conversion failed
zfnet512-12-int8.onnx Conversion failed
zfnet512-12.onnx Converted successfully
zfnet512-3.onnx Conversion failed
zfnet512-6.onnx Conversion failed
zfnet512-7.onnx Conversion failed
zfnet512-8.onnx Conversion failed
zfnet512-9.onnx Converted successfully
ResNet101-DUC-12-int8.onnx Conversion failed
ResNet101-DUC-12.onnx Converted successfully
ResNet101-DUC-7.onnx Conversion failed
FasterRCNN-10.onnx Conversion failed
FasterRCNN-12-int8.onnx Conversion failed
FasterRCNN-12.onnx Conversion failed
fcn-resnet101-11.onnx Converted successfully
fcn-resnet50-11.onnx Converted successfully
fcn-resnet50-12-int8.onnx Conversion failed
fcn-resnet50-12.onnx Converted successfully
MaskRCNN-10.onnx Conversion failed
MaskRCNN-12-int8.onnx Conversion failed
MaskRCNN-12.onnx Conversion failed
retinanet-9.onnx Conversion failed
ssd-10.onnx Conversion failed
ssd-12-int8.onnx Conversion failed
ssd-12.onnx Conversion failed
ssd_mobilenet_v1_10.onnx Conversion failed
ssd_mobilenet_v1_12-int8.onnx Conversion failed
ssd_mobilenet_v1_12.onnx Conversion failed
tinyyolov2-7.onnx Conversion failed
tinyyolov2-8.onnx Conversion failed
tiny-yolov3-11.onnx Conversion failed
yolov2-coco-9.onnx Converted successfully
yolov3-10.onnx Conversion failed
yolov3-12-int8.onnx Conversion failed
yolov3-12.onnx Conversion failed
yolov4.onnx Conversion failed
candy-8.onnx Conversion failed
candy-9.onnx Conversion failed
mosaic-8.onnx Conversion failed
mosaic-9.onnx Conversion failed
pointilism-8.onnx Conversion failed
pointilism-9.onnx Conversion failed
rain-princess-8.onnx Conversion failed
rain-princess-9.onnx Conversion failed
udnie-8.onnx Conversion failed
udnie-9.onnx Conversion failed
super-resolution-10.onnx Converted successfully

Filtering out non-quantised and models that match the min requirements

Open To view Non-quantised, opset 9+ models
model status
bertsquad-10.onnx Conversion failed
bertsquad-12.onnx Conversion failed
bidaf-9.onnx Conversion failed
gpt2-10.onnx Conversion failed
gpt2-lm-head-10.onnx Conversion failed
gpt2-lm-head-bs-12.onnx Conversion failed
roberta-base-11.onnx Converted successfully
roberta-sequence-classification-9.onnx Conversion failed
t5-decoder-with-lm-head-12.onnx Converted successfully
t5-encoder-12.onnx Converted successfully
age_googlenet.onnx Converted successfully
gender_googlenet.onnx Converted successfully
version-RFB-320.onnx Conversion failed
version-RFB-640.onnx Conversion failed
bvlcalexnet-12.onnx Conversion failed
bvlcalexnet-9.onnx Conversion failed
caffenet-12.onnx Conversion failed
caffenet-9.onnx Conversion failed
densenet-12.onnx Converted successfully
densenet-6.onnx Converted successfully
densenet-9.onnx Converted successfully
efficientnet-lite4-11.onnx Converted successfully
googlenet-12.onnx Conversion failed
googlenet-9.onnx Conversion failed
inception-v1-12.onnx Conversion failed
inception-v1-9.onnx Conversion failed
inception-v2-9.onnx Converted successfully
mnist-12.onnx Conversion failed
mobilenetv2-10.onnx Converted successfully
mobilenetv2-12.onnx Converted successfully
rcnn-ilsvrc13-9.onnx Conversion failed
resnet50-caffe2-v1-9.onnx Converted successfully
resnet50-v1-12.onnx Converted successfully
shufflenet-9.onnx Converted successfully
shufflenet-v2-10.onnx Conversion failed
shufflenet-v2-12.onnx Conversion failed
squeezenet1.0-12-int8.onnx Conversion failed
squeezenet1.0-12.onnx Conversion failed
squeezenet1.0-9.onnx Conversion failed
vgg16-12.onnx Converted successfully
vgg19-caffe2-9.onnx Conversion failed
zfnet512-12.onnx Converted successfully
zfnet512-9.onnx Converted successfully
ResNet101-DUC-12.onnx Converted successfully
FasterRCNN-10.onnx Conversion failed
FasterRCNN-12.onnx Conversion failed
fcn-resnet101-11.onnx Converted successfully
fcn-resnet50-11.onnx Converted successfully
fcn-resnet50-12.onnx Converted successfully
MaskRCNN-10.onnx Conversion failed
MaskRCNN-12.onnx Conversion failed
retinanet-9.onnx Conversion failed
ssd-10.onnx Conversion failed
ssd-12.onnx Conversion failed
ssd_mobilenet_v1_10.onnx Conversion failed
ssd_mobilenet_v1_12.onnx Conversion failed
tiny-yolov3-11.onnx Conversion failed
yolov2-coco-9.onnx Converted successfully
yolov3-10.onnx Conversion failed
yolov3-12.onnx Conversion failed
yolov4.onnx Conversion failed
candy-9.onnx Conversion failed
mosaic-9.onnx Conversion failed
pointilism-9.onnx Conversion failed
rain-princess-9.onnx Conversion failed
udnie-9.onnx Conversion failed
super-resolution-10.onnx Converted successfully

43 Failed, 24 Converted

@SuperSecureHuman
Copy link
Contributor

Workaround for opset version

Onnx Version Conversion - Offical Docs

Example
import onnx
from onnx import version_converter
import torch
from onnx2torch import convert

# Load the ONNX model
model = onnx.load("model.onnx")
# Convert the model to the target version
target_version = 13
converted_model = version_converter.convert_version(model, target_version)
# Convert to torch
torch_model = convert(converted_model)
torch.save(torch_model, "model.pt")

This could be added to the docs, and in the error message

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed speedster Issue related to the Speedster App
Projects
None yet
Development

No branches or pull requests

3 participants