Predictions of custom converted YOLOv4 model

[JonathanSamelson]

Ask a Question

Question

Hi. I'm new to inference using ONNX and I'm getting troubles to use the predictions I get using my custom YOLOv4 model, recently converted thanks to this notebook.

Then, I wanted to use the notebook for inference, but I'm not getting the same kind of output. Instead of something like [(1, 52, 52, 3, 85), (1, 26, 26, 3, 85), (1, 13, 13, 3, 85)], I get [(1, 46, 15)].
Next, I get the following error using the postprocess_bbox function:

<ipython-input-3-265b08cda32c> in postprocess_bbbox(pred_bbox, ANCHORS, STRIDES, XYSCALE)
     28         conv_shape = pred.shape
     29         output_size = conv_shape[1]
---> 30         conv_raw_dxdy = pred[:, :, :, :, 0:2]
     31         conv_raw_dwdh = pred[:, :, :, :, 2:4]
     32         xy_grid = np.meshgrid(np.arange(output_size), np.arange(output_size))

IndexError: too many indices for array: array is 3-dimensional, but 5 were indexed

Of course, it does not have the same shape... But I don't understand the meaning of the shape I obtain, knowing that my model output 11 classes and the input resolution is 416, just like the example. In case it helps, my output_names is : [tf.concat_16]

In my custom model, I did not change the anchors, the strides nor the xy scales, it was simply trained with the default values.
Does anyone know the meaning of the shape I get or what's wrong with my model to obtain such a shape?

Is this issue related to a specific model?

Model name (e.g. mnist): YOLOv4 (custom)
Model opset (e.g. 7): 11 (default in the notebook)

[jingyanwangms]

@JonathanSamelson do you get the same error with the onnx conversion of standard yolov4 model?
@jennifererwangg @ravimashru who added those two notebooks who might know more about this model

[JonathanSamelson]

@jingyanwangms Thanks for your answer. No, it works well with the model available at vision/object_detection_segmentation/yolov4/model, I guess that the postprocess functions are meant to work with those outputs.

[ravimashru]

@JonathanSamelson Yes, you're right. That postprocess function works only if the model outputs have the shape [(1, 52, 52, 3, 85), (1, 26, 26, 3, 85), (1, 13, 13, 3, 85)].

[JonathanSamelson]

@ravimashru Thanks. Do you see any reason why I have a shape that is so different even though I have the same model input size (416)? Could it be related to the number of classes (11 in my case)? Or is it related to the conversion...?

[ravimashru]

Honestly, your guess is as good as mine. Is it possible for you to share the code you wrote to create your model in TensorFlow? I can try looking at that to help you figure it out.

[JonathanSamelson]

I did not create the model in Tensorflow but instead converted the .weights file after training on Darknet. Here is the notebook I used for conversion.
Can you try to reproduce it thanks to my trained model? I'll send it to you by email.
Anyways thanks, I appreciate the help!

[AyaNasser96]

Is there any updates ?
I've the same problem.