你好，onnx提供的fasterrcnn简化失败，你能解决一下吗？

[92ypli]

https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn，这是onnx提供的模型，你可以用这个复现，谢谢。

[daquexian]

Could you please provide the error message?

[lucasjinreal]

Simplifying...
2019-09-23 23:08:05.474371187 [E:onnxruntime:, sequential_executor.cc:127 Execute] Non-zero status code returned while running Node: 2 Status Message: Invalid input shape: {0,0}
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.6/dist-packages/onnxsim/__main__.py", line 24, in <module>
    main()
  File "/usr/local/lib/python3.6/dist-packages/onnxsim/__main__.py", line 17, in main
    model_opt = onnxsim.simplify(args.input_model, check_n=args.check_n, perform_optimization=not args.skip_optimization)
  File "/usr/local/lib/python3.6/dist-packages/onnxsim/onnx_simplifier.py", line 200, in simplify
    res = forward_all(model_opt)
  File "/usr/local/lib/python3.6/dist-packages/onnxsim/onnx_simplifier.py", line 107, in forward_all
    res = forward(model)
  File "/usr/local/lib/python3.6/dist-packages/onnxsim/onnx_simplifier.py", line 99, in forward
    res = OrderedDict(zip(outputs, sess.run(outputs, inputs)))
  File "/usr/local/lib/python3.6/dist-packages/onnxruntime/capi/session.py", line 72, in run
    return self._sess.run(output_names, input_feed, run_options)
RuntimeError: Method run failed due to: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Node: 2 Status Message: Invalid input shape: {0,0}

[lucasjinreal]

Seems dynamic input not supported by onnxruntime.

[lucasjinreal]

But actually, onnxruntime can inference model with dynamicly correct

[novioleo]

@jinfagang i think your source code is not correct,please post it.

[lucasjinreal]

It's not about source code. I simply convert maskrcnn.onnx from onnx model zoo. You can try simplifying it. But challenging.

Currently, this model generated with too much complicated node structure, the reason for simpliying is that I want convert it to TensorRT, it's can't be done if not do some model surgeon on it.

[lucasjinreal]

I am able to make it forward by sending numpy.array with certain dimension converted from my input image. For onnx-sim it doesn't handle dynamic input which always get (3,0,0) input size.

I caught some weired Constant op:

input: "605"
output: "913"
name: "911"
op_type: "Constant"
attribute {
  name: "axis"
  i: 0
  type: INT
}

Which seems can not be simplified either can not leave it alone. I can using onnx-sim forward now but can't remove all Constant layers.

When eliminating these nodes will caught error:

elem_type = get_elem_type(model, node.output[0])
            if elem_type != None:
                # print(node)
                # default is TENSOR, INT will not trace here
                shape = res[node.output[0]].shape
                new_attr = onnx.helper.make_attribute(
                    'value',
                    onnx.helper.make_tensor(
                        name=node.output[0],
                        data_type=elem_type,
                        dims=shape,
                        vals=np.array(res[node.output[0]]).flatten().astype(get_np_type_from_elem_type(elem_type))
                    ))
                del node.input[:]
                del node.attribute[:]
                node.attribute.extend(
                    [new_attr])

They have no elem_type from above structure.

[daquexian]

@jinfagang You are right, onnxsim cannot handle dynamic input shape currently, I'll try to support it when I have time

[lucasjinreal]

@daquexian May I ask that from your codes I only saw constant simplifying (clean Const and Eliminate Const) which basically is convert constants with Tensors raw data into a combined Const node. How do u able to simplifying these (shape->gather->unsqueeze) into a single reshape node? Correct me if am wrong.

[daquexian]

@jinfagang great question. As long as the input shape is determined, the output of shape, gather and unsqueeze op will be all marked as const in onnxsim and be replaced by constant ops.

[lucasjinreal]

I see.. so that dynamic inputs graph such as maskrcnn will impossible to simplify in this way... because every middle output shape will be various on different images or same image but different input size.
Do u think there still necessary simplify it in this situation?
The essential point of onnxsim is that I can convert a simed onnx model to tensorrt but before sim it just can not convert. What do u think is the behind root reason for this?

[daquexian]

Do u think there still necessary simplify it in this situation?

Sorry for the late reply. It depends. The ops performed on const data (e.g., weights and bias) will be also eliminated via simplifying, no matter whether the input shape is static.

What do u think is the behind root reason for this?

Could you provide the error log for the failure case? It is helpful to find the reason.

[lucasjinreal]

Some large model can not be or hard to simplify such as maskrcnn.onnx

[daquexian]

Some large model can not be or hard to simplify such as maskrcnn.onnx

Could you please open a separate issue for it? Thanks! This issue is closing since onnxsim now supports setting input shape manually in v0.1.9

[LeonNerd]

https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn
您用的 --input-shape 是什么？谢谢 我想简化后 转为caffe，您有什么好的方案么