--inputs-as-nchw: inputs in NCHW format and outputs in NHWC format

[Patrick-PhoenixAI]

Describe the bug
the --inputs-as-nchw parameter let me feed the model using NCHW format
However the model is outputting NHWC format
by making a transpose at the end of the model.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Unbuntu 18.04
• Tensorflow Version: TensorFlow 1.15
• Python version: 3.6

Expected behavior
Inputs and outputs in NCHW format.

Additional context
I am converting the TensorFlow model on a host PC without GPU but
I am using the ONNX model on an NVIDIA Jetson embedded platform.

[guschmue]

adding the transpose at the output is tricky because the conv is in most cases not the output of the model. Say for resnet it is argmax or softmax.
If we'd have an option to insert a transpose behind a random conv node - that would not work either because the consumers of that conv would not find the data format it expected.
Only if the conv is the output itself we could insert such a transpose but I think those models are very rare.

[srohit0]

If the graph is frozen and ready for inference, wouldn't you transpose parameter (weights and biases) matrices too for a proper conversion from NCHW to NHWC. Once Conv operation is done, one has to convert it back to NCHW, since next layers are expecting NCHW format.

current option --inputs-as-nchw likely assumes that graph is being converted for further training, in that case transposing X is fine, since parameters aren't trained yet.

[guschmue]

--inputs-as-nchw will only add a transpose on the input so you can feed a NCHW when in tf it would be NHWC. For Conv and others we add transposes internally since onnx only understands NCHW. Those transposes we try to optimize away so the final graph will not show them anymore (at least that is the goal).

[srohit0]

For Conv and others we add transposes internally since onnx only understands NCHW

Internally to where?

Since you said ONNX only understands NCHW, I assume it's not internal to ONNX operator. Does that mean, more transpose operators in the generated ONNX graph?

[guschmue]

I meant 'tf2onnx internally transposes everything to NCHW since onnx only supports NCHW'. There is a potential that there are more transposes than in the tensorflow graph but for most models we can optimize them away.

[guschmue]

we assume this is resolved.