I want to generate an onnx with a fixed batch size, but I got an error

[abcdvzz]

Thanks for your error report and we appreciate it a lot.

Checklist

1. I have searched related issues but cannot get the expected help.
2. The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.
when I ran 'python tools/pytorch2onnx.py', and other arguments of course, I can get the correct output of onnx file. But when I change the batchsize of input_shape from 1 to 8(or any other nums > 1), I got an error.
Reproduction

1. What command or script did you run?

   python tools/pytorch2onnx.py (other arguments worked when the input shape is 1,3,h,w)
   

2. Did you make any modifications on the code or config? Did you understand what you have modified?

   if args.shape is None:
       img_scale = cfg.test_pipeline[1]['img_scale']
       input_shape = (8, 3, img_scale[1], img_scale[0])
   elif len(args.shape) == 1:
       input_shape = (8, 3, args.shape[0], args.shape[0])
   elif len(args.shape) == 2:
       input_shape = (
           8,
           3,
       ) + tuple(args.shape)
   else:
       raise ValueError('invalid input shape')
   

I changed the input shape from 1,3,h,w to 8,3,h,w
3. What dataset did you use?
It doesn't matter.
Environment
pytorch 1.6; mmcv 1.3.7; mmseg 0.14.0

1. Please run python mmseg/utils/collect_env.py to collect necessary environment infomation and paste it here.
2. You may add addition that may be helpful for locating the problem, such as
   • How you installed PyTorch [e.g., pip, conda, source]
     pip
   • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)
     no
     Error traceback

Traceback (most recent call last):
  File "tools/pytorch2onnx_old.py", line 212, in <module>
    verify=args.verify)
  File "tools/pytorch2onnx_old.py", line 120, in pytorch2onnx
    opset_version=opset_version)
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/onnx/__init__.py", line 208, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 92, in export
    use_external_data_format=use_external_data_format)
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 530, in _export
    fixed_batch_size=fixed_batch_size)
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 366, in _model_to_graph
    graph, torch_out = _trace_and_get_graph_from_model(model, args)
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 319, in _trace_and_get_graph_from_model
    torch.jit._get_trace_graph(model, args, strict=False, _force_outplace=False, _return_inputs_states=True)
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/jit/__init__.py", line 338, in _get_trace_graph
    outs = ONNXTracedModule(f, strict, _force_outplace, return_inputs, _return_inputs_states)(*args, **kwargs)
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/jit/__init__.py", line 426, in forward
    self._force_outplace,
  File "/workdir/lxj/seg_3.6/lib/python3.6/site-packages/torch/jit/__init__.py", line 415, in wrapper
    out_vars, _ = _flatten(outs)
RuntimeError: Only tuples, lists and Variables supported as JIT inputs/outputs. Dictionaries and strings are also accepted but their usage is not recommended. But got unsupported type numpy.ndarray

Bug fix

If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

[MengzhangLI]

Could you have a look at issue? @RunningLeon

Thank you very much!

Best,

[RunningLeon]

@abcdvzz Hi, could provide the model config file you used and the modification you did if possible? We would like to try to reproduce the issue.
Thanks.

[silverlining21]

I met the same issue..

[RunningLeon]

@silverlining21 Hi, on which backend would you deploy the ONNX model? ONNX Runtime or TensorRT?

[silverlining21]

I think it's not about deploy target， the error occurs while exporting onnx model. I debug the output of type step by step, finally I fix the issue by comment https://github.com/open-mmlab/mmsegmentation/blob/0b11d58773dc83cb4d4e09c0d78e8b2d79a821fe/mmseg/models/segmentors/encoder_decoder.py#L260

[RunningLeon]

@silverlining21 @abcdvzz Hi, all.
Simply changing the following code may not work.

if args.shape is None:
    img_scale = cfg.test_pipeline[1]['img_scale']
    input_shape = (8, 3, img_scale[1], img_scale[0])
elif len(args.shape) == 1:
    input_shape = (8, 3, args.shape[0], args.shape[0])

If you want to deploy on onnxruntime, you can export to ONNX with the dynamic axis on batch and let batch_size be any integer you want in inference time.

[timothylimyl]

@RunningLeon ,

If I want to deploy in TensorRT with static batch size, how do I go about it?

[MeowZheng]

might try to use mmdeploy