how can i use onnx library if my model has more than one input? Please advice what will be the command for that?

[tomshalini]

No description provided.

[TMVector]

What would you like to do with your model? If you're looking to run your model, you will want to use one of the inference frameworks which support ONNX.

For example, with ONNX Runtime:

import onnxruntime as ort
import numpy as np

ort_session = ort.InferenceSession('path/to/your/model.onnx')

outputs = ort_session.run(None, {
    "input_1": np.random.randn(10, 3, 224, 224).astype(np.float32)
    "input_2": np.random.randn(10, 3, 100).astype(np.float32)
})

print(outputs[0])

[tomshalini]

What would you like to do with your model? If you're looking to run your model, you will want to use one of the inference frameworks which support ONNX.

For example, with ONNX Runtime:

import onnxruntime as ort
import numpy as np

ort_session = ort.InferenceSession('path/to/your/model.onnx')

outputs = ort_session.run(None, {
    "input_1": np.random.randn(10, 3, 224, 224).astype(np.float32)
    "input_2": np.random.randn(10, 3, 100).astype(np.float32)
})

print(outputs[0])

I want to convert pytorch model to onnx model first. But my pytorch model has multiple inputs. How should i proceed?

[TMVector]

PyTorch supports exporting to ONNX via their TorchScript or tracing process. You can read their documentation here.

To export multiple a model with multiple inputs, you want to take a look at the documentation for the onnx.export function.

args (tuple of arguments) – the inputs to the model, e.g., such that model(*args) is a valid invocation of the model. ...

So you need to provide the inputs in a tuple, similarly to how you would pass the inputs to the model when running in PyTorch.

If we pretend we had a model with multiple inputs, matching the ONNX model I gave example code for above, it would look like this:

model = TwoInputModel()

dummy_input_1 = torch.randn(10, 3, 224, 224)
dummy_input_2 = torch.randn(10, 3, 100)

# This is how we would call the PyTorch model
example_output = model(dummy_input_1, dummy_input_2)

# This is how to export it with multiple inputs
torch.onnx.export(model,
        args=(dummy_input_1, dummy_input_2),
        f="alexnet.onnx",
        input_names=["input_1", "input_2"],
        output_names=["output1"])

You can see another example here. There is some indirection in that code, so it might look more complicated than it is, but it might help you understand dynamic axes etc.

[tomshalini]

PyTorch supports exporting to ONNX via their TorchScript or tracing process. You can read their documentation here.

To export multiple a model with multiple inputs, you want to take a look at the documentation for the onnx.export function.

args (tuple of arguments) – the inputs to the model, e.g., such that model(*args) is a valid invocation of the model. ...

So you need to provide the inputs in a tuple, similarly to how you would pass the inputs to the model when running in PyTorch.

If we pretend we had a model with multiple inputs, matching the ONNX model I gave example code for above, it would look like this:

model = TwoInputModel()

dummy_input_1 = torch.randn(10, 3, 224, 224)
dummy_input_2 = torch.randn(10, 3, 100)

# This is how we would call the PyTorch model
example_output = model(dummy_input_1, dummy_input_2)

# This is how to export it with multiple inputs
torch.onnx.export(model,
        args=(dummy_input_1, dummy_input_2),
        f="alexnet.onnx",
        input_names=["input_1", "input_2"],
        output_names=["output1"])

You can see another example here. There is some indirection in that code, so it might look more complicated than it is, but it might help you understand dynamic axes etc.

Thank you for your help. I did the same as mentioned in below code:

Code:
from torch.autograd import Variable
import torch
import numpy as np
from model.stereo_rcnn.resnet import resnet

Load the trained model from file

model_path="/home/Models/rcnn_local/models_stereo/stereo_rcnn_12_6477.pth"

stereoRCNN = resnet(('background', 'Car'), 101, pretrained=True)
stereoRCNN.create_architecture()
checkpoint = torch.load(model_path)
stereoRCNN.load_state_dict(checkpoint['model'])

Export the trained model to ONNX

dummy_input_1 = torch.randn(1, 3, 600, 1986).cuda()
dummy_input_2 = torch.randn(1, 3, 600,1986).cuda()
dummy_input_3 = torch.randn(1, 3).cuda()
dummy_input_4 = torch.randn(1,1).cuda()
dummy_input_5 = torch.randn(1,1).cuda()
dummy_input_6 = torch.randn(1,1).cuda()
dummy_input_7 = torch.randn(1,1).cuda()
dummy_input_8 = torch.randn(1,1).cuda()
dummy_input_9 = torch.randn(1,1).cuda()

torch.onnx.export(stereoRCNN.cuda(),
args=(dummy_input_1, dummy_input_2,dummy_input_3, dummy_input_4,dummy_input_5, dummy_input_6,dummy_input_7, dummy_input_8,dummy_input_9),
f="stereorcnn.onnx",
input_names=["im_left_data","im_right_data","im_info","gt_boxes_left","gt_boxes_right","gt_boxes_merge","gt_dim_orien","gt_kpts","num_boxes"],
output_names=["rois_left","rois_right","cls_prob","bbox_pred","dim_orien_pred","kpts_prob","left_border_prob","right_border_prob","rpn_loss_cls","rpn_loss_bbox_left_right","RCNN_loss_cls","RCNN_loss_bbox","RCNN_loss_dim_orien","RCNN_loss_kpts","rois_label"])

Error:
I am getting below error:

File "/home/anaconda3/envs/env_stereo/lib/python3.6/site-packages/torch/nn/modules/module.py", line 492, in call
result = self.forward(*input, **kwargs)
File "/home/anaconda3/envs/env_stereo/lib/python3.6/site-packages/torch/jit/init.py", line 266, in forward
out_vars, _ = _flatten(out)
RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got int

Please advise.

[TMVector]

I have replied on #2679; I think this issue can be closed now @prasanthpul 🙂

[kt-rax]

Add another Q. How about W/ multi output???

[rajumgokak]

input using tensorflow

#path to your model
model_dir="your_path_to_file"
model=model_dir+"/model.onnx"

#Preprocess the image
img=cv2.imread('image_path/img.jpg')
img = cv2.resize(img,dsize=(512,512), interpolation=cv2.INTER_AREA)
img.resize(1,512,512,3)
img = np.array(img).astype('float32')
input11 = img
input22 = ["your input"]
input33 = ['your input"]

session=onnxruntime.InferenceSession(model,None)
input1=session.get_inputs()[0].name
input2 =session.get_inputs()[1].name
input3 =session.get_inputs()[2].name

np.array(input22).flatten()
np.array(input33).flatten()

prediction=session.run([],{input1:input11,input2:input22,input3:input33})
print(prediction)

[peterukk]

Only vaguely related, but in case someone with memory issues during inference stumbles upon this thread: I encountered strange behaviour (a bug?) with ONNX models using multiple inputs (i.e. outputs = session.run(None, { inp_name1: x1, inp_name2: x2,..}). When running inference I had GPU OOM errors at way, way smaller batch sizes than expected. This was fixed by changing my model so that it only had one (larger) array as input (from which I then extracted x1,x2... inside the model). I'm using onnx 1.12.0, tf2onnx 1.14.0 and opset 17.