SparseInst Model exportation to ONNX

[ayoolaolafenwa]

Thank you so much for your work @jinfagang. I have tested yolov7 and I realized that SparseInst models cannot be converted to ONNX. Is the export onnx code compatible with exporting SparseInst models?

[lucasjinreal]

@ayoolaolafenwa Hi, thank u for your interest. Theoretically it should able to export to onnx.

What's error messages did got when export SparseInst?

[ayoolaolafenwa]

I export using
python3 export_onnx.py --config-file configs/coco/sparseinst/sparse_inst_r50vd_giam_aug.yaml --input "cycle.jpg" MODEL.WEIGHTS sparse_inst_r50vd_giam_aug_8bc5b3.pth MODEL.DEVICE cpu

I got this error message
TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results). images = [x["image"].to(self.device) for x in batched_inputs] Traceback (most recent call last): File "export_onnx.py", line 257, in <module> torch.onnx.export(model, inp, onnx_f, output_names={ File "/home/ayoola/.local/lib/python3.8/site-packages/torch/onnx/__init__.py", line 316, in export return utils.export(model, args, f, export_params, verbose, training, File "/home/ayoola/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 107, in export _export(model, args, f, export_params, verbose, training, input_names, output_names, File "/home/ayoola/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 724, in _export _model_to_graph(model, args, verbose, input_names, File "/home/ayoola/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 493, in _model_to_graph graph, params, torch_out, module = _create_jit_graph(model, args) File "/home/ayoola/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 437, in _create_jit_graph graph, torch_out = _trace_and_get_graph_from_model(model, args) File "/home/ayoola/.local/lib/python3.8/site-packages/torch/onnx/utils.py", line 388, in _trace_and_get_graph_from_model torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True) File "/home/ayoola/.local/lib/python3.8/site-packages/torch/jit/_trace.py", line 1166, in _get_trace_graph outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs) File "/home/ayoola/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/home/ayoola/.local/lib/python3.8/site-packages/torch/jit/_trace.py", line 127, in forward graph, out = torch._C._create_graph_by_tracing( File "/home/ayoola/.local/lib/python3.8/site-packages/torch/jit/_trace.py", line 118, in wrapper outs.append(self.inner(*trace_inputs)) File "/home/ayoola/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/home/ayoola/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1090, in _slow_forward result = self.forward(*input, **kwargs) File "/mnt/c/Users/ayoola/Documents/codes/WSL YOLO/yolov7/yolov7/modeling/meta_arch/sparseinst.py", line 93, in forward images = self.preprocess_inputs(batched_inputs) File "/mnt/c/Users/ayoola/Documents/codes/WSL YOLO/yolov7/yolov7/modeling/meta_arch/sparseinst.py", line 65, in preprocess_inputs images = [x["image"].to(self.device) for x in batched_inputs] File "/mnt/c/Users/ayoola/Documents/codes/WSL YOLO/yolov7/yolov7/modeling/meta_arch/sparseinst.py", line 65, in <listcomp> images = [x["image"].to(self.device) for x in batched_inputs] IndexError: too many indices for tensor of dimension 3

[lucasjinreal]

@ayoolaolafenwa please try it again, it should supported now.

[ayoolaolafenwa]

@ayoolaolafenwa please try it again, it should supported now.

Thank you! I will test it!

[wangyidong3]

Hi jinfagang,

Thank you for your excellent work.

When I try to export onnx file, still got "IndexError: too many indices for tensor of dimension 3" , command just as in readme:

python export_onnx.py --config-file configs/coco/sparseinst/sparse_inst_r50_giam_aug.yaml --video-input datasets/video/input.flv  --opts MODEL.WEIGHTS weights/sparse_inst_r50_giam_aug_2b7d68.pth INPUT.MIN_SIZE_TEST 512

Any advice?

[lucasjinreal]

@wangyidong3 If the output says onnx saved into file, then ignore the last error. It was normal since I didn't fully catch the logic verification.

[mattiasbax]

@wangyidong3 @ayoolaolafenwa

Did you have any luck running inference on the exported onnx model? =)
If so, what language and framework did you use for inference and do you have any code you could share?

[JISHNUSHAJI]

@wangyidong3 @ayoolaolafenwa could anyone solve this issue.I am getting the same error.

images = [x["image"].to(self.device) for x in batched_inputs]
IndexError: too many indices for tensor of dimension 3

[lucasjinreal]

Hi, the onnx exported from yolov7-d2, should already included prerpocess, which means, you should not need permute HWC to CHW, and not need to do normalization. it was inside model already.