How to Convert Yolov3 model to CoreML format

[NaeemKhan333]

@john-rocky Thanks for the nice repository. I want to convert the Yolov3 model into core ML format. Can you guide me about the conversion process of yolov3 to coreML? Thank you

[john-rocky]

Hello. @NaeemKhan333

I have not converted Yolov3.
I introduce the article how to convert it.

https://qiita.com/TokyoYoshida/items/68a5f77e72e48fa870e6
https://note.com/wagatuma/n/nea8e29d2e7a7

These are written by Japanese but you can translate by google translate or some and read, I think.

May the force be with you.

[NaeemKhan333]

@john-rocky Thanks for reply , Yes I have tested these articles , model get converted , but when open it in xcode , there is no preview tab to test the results.

In the following article
https://qiita.com/TokyoYoshida/items/68a5f77e72e48fa870e6

The author done following two steps, I do not know where to do this steps , is it done on converted model or is it during training.

5. Examine the display of inference results

The conversion to CoreML worked fine, but when I copy it into my Xcode project and try to infer it in the Vision Framework, it fails.

This is because there are three outputs of YOLOv3, and the shape is 1x1x255x13x13, 1x1x255x26x26, 1x1x255x52x52, but it cannot be interpreted by Vision Framework as it is. You need to decode the output.

This blog was easy to understand about the output of YOLO v3.
Model structure of general object recognition YOLO v3

Decoding seems to be difficult if you make it yourself, so this time I will use this project.

Ma-Dan / YOLOv3-CoreML
https://github.com/Ma-Dan/YOLOv3-CoreML

This project assumes that the CoreML output will be 255x13x13, 255x26x26, 255x52x52.
You need to reshape the output to this shape.

6. Reshape the output

Reshape the output of the model as follows:

1x1x255x13x13 → 255x13x13
1x1x255x26x26 → 255x13x26
1x1x255x52x52 → 255x13x52

To do this, you need to add a layer to reshape with Core ML Tools.
You can read more about how to edit layers in your Core ML model with Core ML Tools here.

How to edit layers with Core ML Tools
https://qiita.com/TokyoYoshida/items/7aa67dcea059a767b4f2

It is a layer to reshape, but at first I tried it because there is add_squeeze that reduces the dimension, but for some reason it did not work.

There was also add_reshape , but this did not reduce the dimension of the first 1x1 part.

As a result of various investigations, there was add_reshape_static , and I was able to reshape it successfully using this.

Add it as follows.

from coremltools.models.neural_network import datatypes

builder.add_reshape_static(name='Reshape1', input_name='grid1', output_name='output1', output_shape=(255,13,13))
builder.add_reshape_static(name='Reshape2', input_name='grid2', output_name='output2', output_shape=(255,26,26))
builder.add_reshape_static(name='Reshape3', input_name='grid3', output_name='output3', output_shape=(255,52,52))

builder.spec.description.output[0].name = "output1"
builder.spec.description.output[0].type.multiArrayType.shape[0] = 255
builder.spec.description.output[0].type.multiArrayType.shape.append(13)
builder.spec.description.output[0].type.multiArrayType.shape.append(13)

builder.spec.description.output[1].name = "output2"
builder.spec.description.output[1].type.multiArrayType.shape[0] = 255
builder.spec.description.output[1].type.multiArrayType.shape.append(26)
builder.spec.description.output[1].type.multiArrayType.shape.append(26)

builder.spec.description.output[2].name = "output3"
builder.spec.description.output[2].type.multiArrayType.shape[0] = 255
builder.spec.description.output[2].type.multiArrayType.shape.append(52)
builder.spec.description.output[2].type.multiArrayType.shape.append(52)

builder.spec.description.output[0].name = "output1"
builder.spec.description.output[0].type.multiArrayType.shape[0] = 255
builder.spec.description.output[0].type.multiArrayType.shape.append(13)
builder.spec.description.output[0].type.multiArrayType.shape.append(13)

builder.spec.description.output[1].name = "output2"
builder.spec.description.output[1].type.multiArrayType.shape[0] = 255
builder.spec.description.output[1].type.multiArrayType.shape.append(26)
builder.spec.description.output[1].type.multiArrayType.shape.append(26)

builder.spec.description.output[2].name = "output3"
builder.spec.description.output[2].type.multiArrayType.shape[0] = 255
builder.spec.description.output[2].type.multiArrayType.shape.append(52)
builder.spec.description.output[2].type.multiArrayType.shape.append(52)

[NaeemKhan333]

@john-rocky There are some steps that are missing or ambiguous in the given links.

https://qiita.com/TokyoYoshida/items/68a5f77e72e48fa870e6 ( Step 6 and 7 are confusing)
https://note.com/wagatuma/n/nea8e29d2e7a7