How to convert mxnet model to caffe?

[yumaofan]

Thanks for your work, Could you share your convert code? I try to convert it use mmdnn, but not work, if you not convenient, Could you give me some direction for it. I don't know how to process crop and upsampling layer. thx again.

[gasgallo]

Interested in the conversion process too...

[yumaofan]

@gasgallo You can attempt ONNX to convert it, MxNet just supports upsampling operator now.
apache/mxnet#15892

[gasgallo]

@AaronFan1992 thanks for the help, actually I've already succeeded in converting to onnx, but for my project I need caffe models unfortunately..

[Zheweiqiu]

@AaronFan1992 thanks for the help, actually I've already succeeded in converting to onnx, but for my project I need caffe models unfortunately..

Did you rebuild mxnet from source to support updampling operator?

[gasgallo]

@Zheweiqiu No, I've manually edited onnx_mxnet

[Zheweiqiu]

@gasgallo Do you mean replace

python/mxnet/contrib/onnx/mx2onnx/_op_translations.py
python/mxnet/contrib/onnx/onnx2mx/_import_helper.py
python/mxnet/contrib/onnx/onnx2mx/_op_translations.py

with the updated version from github?

Update:
I tried update the above three python files but still facing the same error:No conversion function registered for op type UpSampling yet.
Can you provide more details about how you successfully convert mxnet model to onnx or any instructions and reference?
Appreciate for any help!

[gasgallo]

@Zheweiqiu I think master branch still doesn't support many ops (like UpSampling). So you've to code it manually by yourself (that's what I did).

Let me check if I saved my code (I didn't need the onnx model, but I was hoping to be able to convert to Caffe from onnx, but failed). If I do, I'll share it here.

[gasgallo]

Append the following code in _op_translations.py:

@mx_op.register("UpSampling")
def convert_upsample(node, **kwargs):
    """Map MXNet's UpSampling operator attributes to onnx's Upsample operator
    and return the created node.
    """
    name, input_nodes, attrs = get_inputs(node, kwargs)

    sample_type = attrs.get('sample_type', 'nearest')
    sample_type = 'linear' if sample_type == 'bilinear' else sample_type
    scale = convert_string_to_list(attrs.get('scale'))
    scaleh = scalew = float(scale[0])
    if len(scale) > 1:
        scaleh = float(scale[0])
        scalew = float(scale[1])
    scale = [1.0, 1.0, scaleh, scalew]

    node = onnx.helper.make_node(
        'Upsample',
        input_nodes,
        [name],
        scales=scale,
        mode=sample_type,
        name=name
    )
    return [node]

[gasgallo]

I've created a pr here

[Zheweiqiu]

@gasgallo Thanks! One step further after trying your code, "Upsampling" error was gone but new error occurred

File "/opt/conda/lib/python3.7/site-packages/mxnet/contrib/onnx/mx2onnx/_op_translations.py", line 265, in convert_crop
    h, w = kwargs["out_shape"][-2:]
KeyError: 'out_shape'

And I am using here to do the export.
Any idea?

[gasgallo]

@Zheweiqiu Yeah, unfortunately even Crop op is not 100% compatible and I didn't find any stable fix for it yet.

I found two workarounds taht are valid ONLY in the case of using one single scale (1.0):

• remove Crop nodes from the graph because they're useless
• modify Crop in _op_translations.py as follows:

@mx_op.register("Crop")
def convert_crop(node, **kwargs):
    """Map MXNet's crop operator attributes to onnx's Crop operator
    and return the created node.
    """
    name, inputs, attrs = get_inputs(node, kwargs)
    num_inputs = len(inputs)

    y, x = list(parse_helper(attrs, "offset", [0, 0]))
    h, w = list(parse_helper(attrs, "h_w", [0, 0]))
    if name == "crop0":
        border = [x, y, x + 40, y + 40]
    elif name == "crop1":
        border = [x, y, x + 80, y + 80]

    crop_node = onnx.helper.make_node(
        "Crop",
        inputs=[inputs[0]],
        outputs=[name],
        border=border,
        scale=[1, 1],
        name=name
    )

    logging.warning(
        "Using an experimental ONNX operator: Crop. " \
        "Its definition can change.")

    return [crop_node]

I stress again that this works only and only if you will use one single scale equal to 1.0 and the original retinaFace MXNet model as input.

Cheers

[Zheweiqiu]

@gasgallo Model converted. Thank you very much!

[luan1412167]

@Zheweiqiu you can share how to convert it from mxnet to onnx

[luan1412167]

@gasgallo can you share your way convert retinaface mxnet to onnx ?

[Zheweiqiu]

@luan1412167 apache/mxnet#15892 and #4 may help you out. Post it if you encounter any problem and I'll try to answer it if I get a clue.

[WIll-Xu35]

Also interested in converting retinaface to caffe. Seems like this issue is mostly discussing mxnet to onnx conversion. Any idea how to do the mnxet to caffe converion? Many thanks.

[gasgallo]

@WIll-Xu35 use this

[WIll-Xu35]

@gasgallo Thanks, I'll give it a try.

[ethanyhzhang]

@Zheweiqiu After handling Crop issue, I met a new problem as below:
onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: spatial for operator BatchNormalization
Do you have any idea?

[yumaofan]

Guys, I solved it recently, for Retinaface, you can remove Crop layer directly, and using Deconvolution instead of Upsampling. retrain the model using mxnet, can be perfect convert to caffe.

[PRYUS]

@Zheweiqiu After handling Crop issue, I met a new problem as below:
onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: spatial for operator BatchNormalization
Do you have any idea?

Try working with onnx version 1.3.0. Because BatchNormalization support was dropped after that.

[mohamadHN93]

Append the following code in _op_translations.py:

@mx_op.register("UpSampling")
def convert_upsample(node, **kwargs):
    """Map MXNet's UpSampling operator attributes to onnx's Upsample operator
    and return the created node.
    """
    name, input_nodes, attrs = get_inputs(node, kwargs)

    sample_type = attrs.get('sample_type', 'nearest')
    sample_type = 'linear' if sample_type == 'bilinear' else sample_type
    scale = convert_string_to_list(attrs.get('scale'))
    scaleh = scalew = float(scale[0])
    if len(scale) > 1:
        scaleh = float(scale[0])
        scalew = float(scale[1])
    scale = [1.0, 1.0, scaleh, scalew]

    node = onnx.helper.make_node(
        'Upsample',
        input_nodes,
        [name],
        scales=scale,
        mode=sample_type,
        name=name
    )
    return [node]

Hi I attached your function to the _op_translations.py but got these errors
Traceback (most recent call last):
File "", line 7, in
File "C:\Users\mh.nakhaei\PycharmProjects\untitled\venv\lib\site-packages\mxnet\contrib\onnx\mx2onnx\export_model.py", line 79, in export_model
verbose=verbose)
File "C:\Users\mh.nakhaei\PycharmProjects\untitled\venv\lib\site-packages\mxnet\contrib\onnx\mx2onnx\export_onnx.py", line 249, in create_onnx_graph_proto
idx=idx
File "C:\Users\mh.nakhaei\PycharmProjects\untitled\venv\lib\site-packages\mxnet\contrib\onnx\mx2onnx\export_onnx.py", line 86, in convert_layer
raise AttributeError("No conversion function registered for op type %s yet." % op)
AttributeError: No conversion function registered for op type UpSampling yet.