Converting SSD MobileNet V2 tensorflow frozen graph .pb file to onnx model and quantizing it using Onnx Runtime

[The-ButterflyEffect]

``Hi, I am trying to convert SSD Mobilenet V2 tensorflow model ( frozen graph .pb format) to onnx.
I see group attr of the first Conv op in the converted onnx model equal to 0.
This invalid "group=0" then throws error when I attempt to quantize this onnx model using onnxruntime.

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Conv node. Name:'Conv__8004' Status Message: Input channels C is not equal to kernel channels * group. C: 3 kernel channels: 3 group: 0

Used the following python script for conversion:

import onnx
import tf2onnx
import tensorflow as tf

shape_dict = {"image_tensor:0": (3,300,300,3)}
tf_compat_v1 = tf.compat.v1
with tf_compat_v1.gfile.GFile("./frozen_inference_graph.pb", "rb") as f:
    graph_def = tf_compat_v1.GraphDef()
    graph_def.ParseFromString(f.read())

onnx_model, _ = tf2onnx.convert.from_graph_def(graph_def, input_names=["image_tensor:0"], output_names=["num_detections:0", "detection_boxes:0", "detection_scores:0", "detection_classes:0"], opset=11, shape_override=shape_dict)
onnx.save(onnx_model, "model.onnx")

Got the frozen ssd mobilenet v2 model from here: https://gist.github.com/qianlin404/c7991048c5f3264ce13d520ce0274493

[hwangdeyu]

Could you also provide the repro script code running the model.onnx by onnxruntime?

[The-ButterflyEffect]

Hi @hwangdeyu following is the repro script code running the model.onnx by onnxruntime.

`import onnxruntime.backend
import onnx
import tf2onnx
import tensorflow as tf
import numpy as np
shape_dict = {"image_tensor:0": (3,300,300,3)}
tf_compat_v1 = tf.compat.v1
with tf_compat_v1.gfile.GFile("./frozen_inference_graph.pb", "rb") as f:
graph_def = tf_compat_v1.GraphDef()
graph_def.ParseFromString(f.read())

onnx_model, _ = tf2onnx.convert.from_graph_def(graph_def, input_names=["image_tensor:0"], output_names=["num_detections:0", "detection_boxes:0", "detection_scores:0", "detection_classes:0"], opset=11, shape_
onnx.save(onnx_model, "model.onnx")
x_shape = (3,300,300,3)
x_array = np.random.randint(low=0, high=255, size=x_shape).astype("uint8")
inputs = [x_array]
rep = onnxruntime.backend.prepare(onnx_model.SerializeToString(), "CPU")
if isinstance(inputs, list) and len(inputs) == 1:
inp = inputs[0]
else:
inp = inputs
output = rep.run(inp)`

[The-ButterflyEffect]

I have attached a part of the onnx model that I get after .pb to .onnx conversion using tf2onnx. I observe that the onnx graph is broken into two group. That doesn't seem a correct conversion and might be creating the issue of group attribute = 0 in the first conv op. Could you please point out if I am making some mistake in the conversion from .pb to .onnx?
The script for conversion is mention above.
Thanks!

[guschmue]

If I convert it with
python -m tf2onnx.convert --input ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb --output /tmp/t.onnx --inputs image_tensor:0 --outputs num_detections:0,detection_boxes:0,detection_scores:0,detection_classes:0 --opset 13
it is fine.

Works with your code as well.

I used the latest tf2onnx-1.9.3 but don't recall seeing such error before.

Btw, for this model you don't need shape_dict - pretty rare that you would pass this in but it is not the source of the issue.

[The-ButterflyEffect]

• Thanks @guschmue , I checked with 1.9.3 it works fine.

• I had 1.8.5 in my environment.

• with 1.9.3, I no longer see group = 0 for the conv op hence this error no longer shows. "Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Conv node. Name:'Conv__8004' Status Message: Input channels C is not equal to kernel channels * group. C: 3 kernel channels: 3 group: 0"

• Regarding the "shape_dict". I have a requirement to get the onnx model with fixed shape input.

However, I am not able to understand the graph that I get. Which is still broken into two as uploaded before. (Now the conv op has group = 1 attr that is a valid value) Does this mean my onnx graph has no dependency on the input at all??

[guschmue]

this model is trained for the input shape (None,None,None,3), aka it would internally do the right thing for any batch size, h and w. tf2onnx will find the shapes, pretty rare that you need to override it.

[The-ButterflyEffect]

this model is trained for the input shape (None,None,None,3), aka it would internally do the right thing for any batch size, h and w. tf2onnx will find the shapes, pretty rare that you need to override it.

I agree that it is not needed to specify the shape. But, does specifying the shape can cause this resultant graph?, i.e., disconnected input? I mean the trained model should get the inputs fed to it. It is not just the matter of input shapes(which I am overriding) but input data as well, right?

[guschmue]

oh, you mean the image_tensor:0->cast nodes? They are not really disconnected. They are consumed in a subgraph (ie the tf2onnx__15 subgraph) but netron does not show this nicely.

[The-ButterflyEffect]

oh, you mean the image_tensor:0->cast nodes? They are not really disconnected. They are consumed in a subgraph (ie the tf2onnx__15 subgraph) but netron does not show this nicely.

I see! Thanks a lot for mentioning it. Could you please elaborate this part "ie the tf2onnx__15 subgraph". I am not sure what does this mean.
Thanks!

[guschmue]

tf2onnx_15 is the body of the loop. I think the model is taking the batches apart. In netron there is a menu for the subgraphs if you click on the model input or output.

[The-ButterflyEffect]

Did you mean like this "tf2onnx__55" shown in image as a part of Loop "body" Attributes?

[The-ButterflyEffect]

tf2onnx_15 is the body of the loop. I think the model is taking the batches apart. In netron there is a menu for the subgraphs if you click on the model input or output.

Thank you @guschmue

[fatcat-z]

Close this since the issue has been addressed.