EdgeTPU - ResizeBilinear only for small Models?

[walidproggen]

Hello,

i want to use Google Coral Accelerator for Semantic Segmentation. Most Networks for Semantic Segmentation uses Encoder/Decoder architecture to accomplish such tasks. My goal is to run Unet on the edgetpu (coral usb accelerator).

Ive started with a very simple Keras/Tensorflow model like this:

    input_layer = Input(shape=(512, 512, 3))
    x = Conv2D(32, 3, padding = 'same')(input_layer)
    x = BatchNormalization()(x)
    x = Activation('relu')(x)   
    x = MaxPooling2D()(x)
    x = UpSampling2D(interpolation='bilinear')(x)
    
    output_layer = Conv2D(1, 1, padding = 'same', activation = 'sigmoid')(x)
    model = Model(inputs = input_layer, outputs = output_layer)
    model.compile(optimizer = Adam(lr = 0.001), loss = 'binary_crossentropy', metrics = ['accuracy'])

After converting this model to tflite (with Tensorflow 1.15.0 Version) i tried to convert this model to edgetpu. Unfortunally for the ResizeBilinear Operator i get this message:

Operation is otherwise supported, but not mapped due to some unspecified limitation

When i change the Input Shape to smaller size, for example 128x128x3, the ResizeBilinear Operator mapps perfectly to edgetpu and i can smoothly run the compiled model on edgetpu.

On coral.ai it says for ResizeBilinear:

Input/output is a 3-dimensional tensor. Depending on input/output size, this operation may not be mapped to the Edge TPU to avoid loss in precision.

So my question is:
Is there a way to force mapping UpSampling2D/ResizeBilinear to Edgetpu despite input/output size?

Otherwise i see no use case for edgetpu in semantic segmentation. (What i have seen is that DeepLabV3 seems to work according Benchmark of Google, but we get no information about which project exactly was used, so we cant use it for custom data).

One more question: Even if ResizeBilinear is not mapped to edgetpu, i expect the model to run on edgetpu + cpu. But if i run the compiled tflite file, i get the Error:

RuntimeError: Internal: :71 tf_lite_type != kTfLiteUInt8 (9 != 3)Node number 5 (EdgeTpuDelegateForCustomOp) failed to prepare.

Im running it with the Code-Example provided by Google: https://github.com/google-coral/tflite.git
Under tflite/python/examples/classification/classify_image.py
I use this code for my model.

[Naveen-Dodda]

Hello,

I am glad to see application of egde tpu in image segmetation. Yes, ResizeBilinear operation is mapped to cpu in specific cases. As we cannot disclose how this happeens, i can tell you why. Compiler rejects mapping the ops to edgetpu if loss of precision is higher than threshold. Team is working to resolve this and will be updated in our next release.

The reason for runtime errro could be due diffrent vresion of tensorflow. We recommned to use tf 1.15. Please check it, if you are using tf 2.0 or tf 1.15 nightly try to downgrade and run the model.

Hope this helps

Thanks

Best,
Naveen Dodda

[walidproggen]

Hello Naveen,

im glad to hear that you are working on this problem for the next realese version.
It will be a huge advantage for semantic segmentation models.

Regarding the other problem (RuntimError when mapping to cpu), i am using Tensorflow Version 1.15.0 by installing it with:
pip install tensorflow==1.15.0

But the problem still exists. What else can cause the RuntimeError ?

[rul4]

Hi all,

I am also waiting for this fix, any timeframe to the next release? End of the month? Next month? No idea?

Thanks!

[bhack]

Gently ping

[bhack]

Any news with the new compiler?
#45

[Naveen-Dodda]

Hi,

We have updated the compiler. https://coral.ai/news/updates-01-2020/

Thanks for your patience

[Naveen-Dodda]

Please feel free to open this if you have more questions. I am closing this as there is not activity.

Thanks

[Tridet]

I feel the main issue was not resolved in the latest update: when using ResizeBilinear in a model, edgetpu_compiler will not map it to TPU if size of input is "too big" (what does it even mean?), and the users don't have a way to prevent this from happening.