YOLACT conversion error support

[taisuke-tomida]

ERROR: input_layer1 layer_id=616: KerasTensor(type_spec=TensorSpec(shape=(1, 19248, 2), dtype=tf.float32, name=None), name='tf.math.multiply_1/Mul:0', description="created by layer 'tf.math.multiply_1'")
ERROR: The trace log is below.
Traceback (most recent call last):
  File "openvino2tensorflow.py", line 474, in convert
    tmp_layers = [tf_layers_dict[from_layer_id].transpose(0,2,3,1).astype(np.float32) if type(tf_layers_dict[from_layer_id]) == np.ndarray else tf_layers_dict[from_layer_id] for from_layer_id in get_tf_edges_from(tf_edges, layer_id)]
  File "openvino2tensorflow.py", line 474, in <listcomp>
    tmp_layers = [tf_layers_dict[from_layer_id].transpose(0,2,3,1).astype(np.float32) if type(tf_layers_dict[from_layer_id]) == np.ndarray else tf_layers_dict[from_layer_id] for from_layer_id in get_tf_edges_from(tf_edges, layer_id)]
ValueError: axes don't match array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "openvino2tensorflow.py", line 477, in convert
    tmp_layers = [tf_layers_dict[from_layer_id].transpose(0,2,3,1) if type(tf_layers_dict[from_layer_id]) == np.ndarray else tf_layers_dict[from_layer_id] for from_layer_id in get_tf_edges_from(tf_edges, layer_id)]
  File "openvino2tensorflow.py", line 477, in <listcomp>
    tmp_layers = [tf_layers_dict[from_layer_id].transpose(0,2,3,1) if type(tf_layers_dict[from_layer_id]) == np.ndarray else tf_layers_dict[from_layer_id] for from_layer_id in get_tf_edges_from(tf_edges, layer_id)]
ValueError: axes don't match array

[PINTO0309]

Please transcribe all error messages here without omitting them. Also, please share the model you are converting from. This information does not tell us anything.

[PINTO0309]

Closed due to lack of progress.

[taisuke-tomida]

I am sorry for the delay.
I am trying to convert "yolact" to tflite, and would like to output a model that includes the NMS processing of yolact. ONNX was able to include NMS. In addition, we have completed the conversion to OpneVINO IR.

command

python openvino2tensorflow.py --model_path yolact_detect_pytorchnms11.xml --model_output_path yolact_detect_pynms --output_saved_model --output_pb --output_no_quant_float32_tflite --output_weight_quant_tflite --output_integer_quant_tflite

Result

�[07mTensorFlow/Keras model building process starts�[0m ======================================
ERROR: axes don't match array
ERROR: model_path  : yolact_detect_pytorchnms11.xml
ERROR: weights_path: yolact_detect_pytorchnms11.bin
ERROR: layer_id    : 617
ERROR: input_layer0 layer_id=528: Const(ndarray).shape  (1, 19248, 2)
array([[[0.00724638, 0.00724638],
        [0.00724638, 0.00724638],
        [0.00724638, 0.00724638],
        ...,
        [0.9       , 0.9       ],
        [0.9       , 0.9       ],
        [0.9       , 0.9       ]]], dtype=float32)
ERROR: input_layer1 layer_id=616: KerasTensor(type_spec=TensorSpec(shape=(1, 19248, 2), dtype=tf.float32, name=None), name='tf.math.multiply_1/Mul:0', description="created by layer 'tf.math.multiply_1'")
ERROR: The trace log is below.
Traceback (most recent call last):
  File "openvino2tensorflow.py", line 474, in convert
    tmp_layers = [tf_layers_dict[from_layer_id].transpose(0,2,3,1).astype(np.float32) if type(tf_layers_dict[from_layer_id]) == np.ndarray else tf_layers_dict[from_layer_id] for from_layer_id in get_tf_edges_from(tf_edges, layer_id)]
  File "openvino2tensorflow.py", line 474, in <listcomp>
    tmp_layers = [tf_layers_dict[from_layer_id].transpose(0,2,3,1).astype(np.float32) if type(tf_layers_dict[from_layer_id]) == np.ndarray else tf_layers_dict[from_layer_id] for from_layer_id in get_tf_edges_from(tf_edges, layer_id)]
ValueError: axes don't match array

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "openvino2tensorflow.py", line 477, in convert
    tmp_layers = [tf_layers_dict[from_layer_id].transpose(0,2,3,1) if type(tf_layers_dict[from_layer_id]) == np.ndarray else tf_layers_dict[from_layer_id] for from_layer_id in get_tf_edges_from(tf_edges, layer_id)]
  File "openvino2tensorflow.py", line 477, in <listcomp>
    tmp_layers = [tf_layers_dict[from_layer_id].transpose(0,2,3,1) if type(tf_layers_dict[from_layer_id]) == np.ndarray else tf_layers_dict[from_layer_id] for from_layer_id in get_tf_edges_from(tf_edges, layer_id)]
ValueError: axes don't match array

Yolact: https://github.com/dbolya/yolact

[PINTO0309]

Converted to TFLite. There is a bug, so I will fix it and commit it this evening.

[PINTO0309]

Related issue: #37

[PINTO0309]

When converted to OpenVINO, there are four outputs.

[PINTO0309]

Fixes.
https://github.com/PINTO0309/openvino2tensorflow/releases/tag/v1.15.3

[taisuke-tomida]

Thank you!!

[PINTO0309]

https://docs.openvinotoolkit.org/2021.4/openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_YOLACT.html

[taisuke-tomida]

I've tried it before too.
Is it still difficult to include Non Max Supression in the network?

[PINTO0309]

You can. If NMS is included in the original ONNX. I just haven't worked that far into the process because it's too much trouble.

[PINTO0309]

Alternatively, you can manually add the NMS to the end of the .xml. But it is very tedious.

[taisuke-tomida]

I was able to use torchvison.ops.nms to include NMS in ONNX. Can I give you the ONNX file personally?

[PINTO0309]

Also, please share the model you are converting from.

At the beginning of this issue I had asked you to do that.

[taisuke-tomida]

I've just deployed the model in an email to you.