[ShapeInferenceError] Input tensor must have at least 2 dimensions

[wwdok]

Hello, @rajeevsrao @pranavm-nvidia @borisfom, i am using onnx-graphsurgeon to isolate a subgraph, this is the code:

import onnx_graphsurgeon as gs
import numpy as np
import onnx

model = onnx.load("model_float32.onnx")
graph = gs.import_onnx(model)
tensors = graph.tensors()
graph.inputs = [tensors["input_1"].to_variable(dtype=np.float32) ]
graph.outputs = [tensors["output_mesh_identity"].to_variable(dtype=np.float32),tensors["conv_faceflag"].to_variable(dtype=np.float32)]
graph.cleanup()
onnx.save(gs.export_onnx(graph), "subgraph1.onnx")

this is the original onnx model:
model_float32.zip

this is the exported subgraph model:
subgraph1.zip

then i try to run the exported model with following code:

import onnxruntime
import numpy as np

model_path = 'subgraph1.onnx'
dummy_input = np.ones([1,192,192,3], dtype=np.float32)

session_option = onnxruntime.SessionOptions()

session = onnxruntime.InferenceSession(
    model_path,
    session_option,
    providers = [
        "CPUExecutionProvider"
    ]
)
input_name = session.get_inputs()[0].name
output_names = [o.name for o in session.get_outputs()]
input_shape = session.get_inputs()[0].shape
output = session.run(
    output_names,
    {input_name: dummy_input.transpose((0, 3, 1, 2))}
)

it reports error:

Traceback (most recent call last):
  File "c:\MachineLearning\PythonScripts\temp.py", line 13, in <module>
    "CPUExecutionProvider"
  File "C:\ProgramData\Anaconda3\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 335, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "C:\ProgramData\Anaconda3\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 370, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from subgraph1.onnx failed:Node (Conv__840) Op (Conv) [ShapeInferenceError] Input tensor must have atleast 2 dimensions

[pranavm-nvidia]

@wwdok When you use to_variable, it's erasing the tensor shape. Since those tensors are already variables, you can omit the call to to_variable.

This works for me:

$ polygraphy surgeon extract model_float32.onnx -o subgraph.onnx \
    --outputs output_mesh_identity:auto conv_faceflag:auto
...
[I] Saving ONNX model to: subgraph.onnx
[I] New Model:
    Name: tf2onnx | ONNX Opset: 11
    
    ---- 1 Graph Input(s) ----
    {input_1 [dtype=float32, shape=(1, 3, 192, 192)]}
    
    ---- 2 Graph Output(s) ----
    {output_mesh_identity [dtype=float32, shape=(1, 1, 1, 1404)],
     conv_faceflag [dtype=float32, shape=(1, 1, 1, 1)]}
    
    ---- 89 Initializer(s) ----
    
    ---- 209 Node(s) ----

$ polygraphy run subgraph.onnx --onnxrt

[I] onnxrt-runner-N0-05/13/22-09:25:05  | Activating and starting inference
[I] Creating ONNX-Runtime Inference Session with providers: ['CPUExecutionProvider']
[I] onnxrt-runner-N0-05/13/22-09:25:05 
    ---- Inference Input(s) ----
    {input_1 [dtype=float32, shape=(1, 3, 192, 192)]}
[I] onnxrt-runner-N0-05/13/22-09:25:05 
    ---- Inference Output(s) ----
    {output_mesh_identity [dtype=float32, shape=(1, 1, 1, 1404)],
     conv_faceflag [dtype=float32, shape=(1, 1, 1, 1)]}
[I] onnxrt-runner-N0-05/13/22-09:25:05  | Completed 1 iteration(s) in 5.171 ms | Average inference time: 5.171 ms.
[I] PASSED | Command: polygraphy run subgraph.onnx --onnxrt