Dynamic Input Reshape Incorrect

[peiwenhuang27]

Describe the bug
I have a trained Tensorflow model that has two inputs:

input0 shape: (64, 60, 257)
input1 shape: (64, 257, 60, 1)

Then I converted it to ONNX model via tf2onnx with the command

# keras model
!python -m tf2onnx.convert \
--saved-model ./model/myModel \
--opset 11 \
--tag serve \
--signature_def serving_default \
--output ./model/myModel.onnx

and use it for inference with dynamic input shape by the following code:

model = onnx.load(ONNX_PATH)
# input0: 64, 60, 257 -> N, N, 257
# input1: 64, 257, 60, 1 -> N, 257, N, 1

model.graph.input[0].type.tensor_type.shape.dim[0].dim_param = 'batch_size'
model.graph.input[0].type.tensor_type.shape.dim[1].dim_param = 'seq_len'
model.graph.input[1].type.tensor_type.shape.dim[0].dim_param = 'batch_size'
model.graph.input[1].type.tensor_type.shape.dim[2].dim_param = 'seq_len'

ONNX_PATH = './model/dynamicModel.onnx'
onnx.save(model, ONNX_PATH)

My inference code:

so = onnxruntime.SessionOptions()
so.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
ort_session = onnxruntime.InferenceSession(ONNX_PATH, sess_options=so)
ort_session.set_providers(['CPUExecutionProvider'])
# run inference
preds = ort_session.run([output_name], {input_name0: x[:32], input_name1: x_norm[:32]})

If I pass inputs as (64, 60, 257) and (64, 257, 60, 1), the session runs fine.
But when I tried to pass inputs shaped as (32, 60, 257) and (32, 257, 60, 1), I get the following error:

RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Reshape node. 
Name:'StatefulPartitionedCall/model/1/Conv2D__36' 
Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:42 
onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, std::vector<long int>&) gsl::narrow_cast<int64_t>(input_shape.Size()) == size was false. 
The input tensor cannot be reshaped to the requested shape. Input shape:{32,257,60,1}, requested shape:{64,1,257,60}

Urgency
I'm trying to keep up with my project deadline on Aug. 13.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Google Colab
• Tensorflow version: 2.5
• Python version: 3.7
• Onnx runtime version: 1.6
• Onnx version: 1.7

To Reproduce
Describe steps/code to reproduce the behavior. Please upload/link the model you are trying to convert if possible.
model link

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
The error occurs in the first operation Reshape as shown in this picture

The reshape op is supposed to reshape input of (64, 257, 60, 1) into (64, 1, 257, 60).
But the error message says Input shape:{32,257,60,1}, requested shape:{64,1,257,60}, which means the tensor was not reshaped correctly.

[TomWildenhain-Microsoft]

Copying over response from microsoft/onnxruntime#8591:

You can't just change the input shape by modifying the graph like that once it is converted. There can be many ops within the graph that depend on the input shape matching what was initially declared. You'll need to override the shape before converting to onnx. Can you edit the tf model signature? You can also override the shape with tf2onnx's --input flag use -1 to indicate unknown dims