Improve semantic of TransposeConv

[zhenhuaw-me]

Serve as a list of trivial items that are not tracked by other issues.

• Merge all trivial activation into one operator converter class
• Adjust createVector() such that the data type of the created tensor is parsed from the NDArray rather than the ref tensor.
• Have a util function to create empty tensor for optional inputs of ONNX operator
• assert with TypeMapping to check TFLite opcode
• Minor comment update for Resize

[zhenhuaw-me]

TODO of TransposeConv (due to we do not fully understand the semantic gap) #21 (comment)

1. Special output shape for VALID padding
2. Different input/output shape for SAME padding

[zhenhuaw-me]

changed the title to track the TransposeConv issue, will not track others anymore.

[zhenhuaw-me]

all done

[ed-mr]

Is there any follow-up on this issue? Since I am not happy about the current decision, that onnx is not supporting NHWC order for half its operators. For all the folks using Conv and TF/Keras it would be the strait thing to support NHWC as onnx attribute in storage_order in the first right in the first place

[zhenhuaw-me]

@ed-mr I am sorry but what's the contex are you refering to?

[ed-mr]

just a sort summary. I just stumbled over this old already closed issue. Which I would like to have questions about. Since to my understanding, it would be much better to improve onnx and provide the "storage_order" attribute to conv and gain abstraction. That way, tf2onnx can easily use its default data_format = "columns_last".
see details on the issue I raised ealier; also onnx/tensorflow #1910 and onnx #4099
I would be happy to see this, since it resolves issues on the backend sides. (I working with tvm)

[zhenhuaw-me]

Yeah, I absolutely agree that ONNX needs to improve for these layout issues. I like the idea of storage_order and I think I have seen somewhere there is related discussions about supporting different layouts. But that would require the ONNX community to drive which is out of the scope of this project :)

I doubt that won't happen soon since the ONNX community would prioritize other things - it's not a blocking issue and we can always workaround it...

[ed-mr]

Besides the fact, that I would be in favour or the storage_order, I figure from the discussion I raised in the ONNX community #4099, the onnx-people might pick up such issue as local functions as described in #3877. Reason sounds ok, they claim that ONNX operator basis becomes to big, since there are many operators to be adopted. I as a small user, and not a contributor I am certainly not able to convince anybody on that end. It might need some involvement from this project. Only issue I see is, to get the named operators NHWCConv on the list mentioned.

[zhenhuaw-me]

@ed-mr Thanks, let me commont on the thread you initialized.