How to convert tensorflow pb (in NCHW format with ResizeOp) into OpenVino IR?

[ausk]

Hi, I have a tensorflow pb model in NCHW format，how to convert it into OpenVino IR?

If I convert it directly, then I get

<layer id="0" name="image_input" type="Parameter" version="opset1">
    <data element_type="f16" shape="1,224,1,224"/>
    <output>
        <port id="0" precision="FP16">
            <dim>1</dim>
            <dim>224</dim>
            <dim>1</dim>
            <dim>224</dim>
        </port>
    </output>
</layer>

What openvino thinks: nhwc(1,1,224,224) => nchw(1,224,1,224)
But actually, it is: nchw(1,1,224,224) => nwch(1,224,1,224).

While there is no predefined NWCH format in openvino.

So does anyone know how to convert the pb(in NCHW format) into IR?

[ilyachur]

@lazarevevgeny Can you help?

[lazarevevgeny]

If your TF model is in NCHW layout you need to specify --disable_nhwc_to_nchw command line parameter when you convert the model.

[ausk]

@ilyachur @lazarevevgeny Thank you guys.

I convert the pb( NCHW) with option --disable_nhwc_to_nchw.

And when I run infer use openvino ( just in CPU first), I get one error:

Interpolate operation should be converted to Interp.

The openvino info:

 Device info:                CPU
        MKLDNNPlugin version ......... 2.1
        Build ........... 42025

---

My pb model contain Node ResizeNearNeighbor. If I convert the pb with --disable_nhwc_to_nchw, then I get a IR with such desc:

<layer id="285" name="fpn0/ResizeNearestNeighbor/Cast_125703_const" type="Const" version="opset1">
    <data element_type="i64" offset="47474976" shape="2" size="16"/>
    <output>
        <port id="1" precision="I64">
            <dim>2</dim>
        </port>
    </output>
</layer>
<layer id="286" name="fpn0/ResizeNearestNeighbor" type="Interpolate" version="opset1">
    <data align_corners="0" antialias="0" axes="1,2" mode="nearest" pads_begin="0" pads_end="0"/>
    <input>
        <port id="0">
            <dim>1</dim>
            <dim>14</dim>
            <dim>14</dim>
            <dim>128</dim>
        </port>
        <port id="1">
            <dim>2</dim>
        </port>
    </input>
    <output>
        <port id="2" precision="FP16">
            <dim>1</dim>
            <dim>28</dim>
            <dim>28</dim>
            <dim>128</dim>
        </port>
    </output>
</layer>

Then what should I do?

[ausk]

I make a small pb model (in OpenVino 2020.R2):

myresize_nchw.zip

You can test with it.

[ilyachur]

@ausk Thank you for the reproducer.

I checked it on the last master and the problem is that at the current moment OpenVINO supports Interpolate operation which is performed over spatial dimensions only.

Unfortunately in your case Interpolate is performed on axises 1, 2.

[ilyachur]

We will look to this problem and find solution to fix it.

[lazarevevgeny]

@ilyachur it is a different story. The MO does not convert the model correctly. During the conversion the Transpose before ResizeNearestNeighbor should be removed because in OpenVINO Interpolate works in NCHW layout but in TF the ResizeNearestNeighbor expects input in the NHWC layout. This is a tricky bug to be solved on the MO side.

[ausk]

@ilyachur @lazarevevgeny Thank you for your help. Best regards.

[lazarevevgeny]

@ausk, the issue has been fixed. Please, take the MO from the master branch and convert the model again. Then use IE from OpenVINO 2020.4 to infer the model. Closing the ticket.

[witignite]

@ausk Could you let me know how you created a nice graph figure from the pb model?

[lazarevevgeny]

@witignite , you can use Netron to visualize the graph.