MobileDet EdgeTPU model lacks post-processing step

[sinclairnick]

Prerequisites

N/A
Please answer the following question for yourself before submitting an issue.

• I checked to make sure that this feature has not been requested already.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md#pixel4-edge-tpu-models

2. Describe the feature you request

The CPU version of MobileDet has post-processing, but the EdgeTPU version does not. Moreover, I have been unable to convert the frozen graph to tflite to add the post-processing step via the provided export_ssd_tflite.py script with --add_postprocessing_op option. In short, I would like to request a MobileDet model with post-processing op for EdgeTPU, please.

3. Additional context

N/A

4. Are you willing to contribute it? (Yes or No)

Yes. I'm happy to convert it myself, but despite investigating the source code, I've been unable to reproduce the post-processing step and unable to convert the model to TFLite.

[Namburger]

Hi team,
would also love to know how you guys were able to quantized the model?
I tried quantizing the cpu version which seems to have the post processing ops. I used both the tflite_convert:

$ tflite_convert --output_file=out.tflite --graph_def_file=tflite_graph.pb  \
--input_shapes=1,320,320,3 \
--input_arrays normalized_input_image_tensor \ 
--output_arrays "TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3" \
--mean_values 128 \ 
--std_dev_values 128 \
--allow_custom_ops \
--change_concat_input_ranges=false \
--allow_nudging_weights_to_use_fast_gemm_kernel=true

and this simple POC script to see if it would quantize, but keep getting some issue related to this: tensorflow/tensorflow#22994

Any pointers would be highly appreciated!

[Namburger]

@sinclairnick
I know it's been a while and not sure if you're still interested. But after looking a little more closely, I realized that the model was just missing the post processing ops.
Using export_tflite_ssd_graph.py with --add_postprocessing_op=true was enough to add the post processing + nms to the model:

$ python3 ~/models/research/object_detection/export_tflite_ssd_graph.py \
--pipeline_config_path=pipeline.config \
--trained_checkpoint_prefix=model.ckpt-400000 \
--output_directory=out \
--add_postprocessing_op=true

Then convert it to tflite format:

$ tflite_convert --output_file=out.tflite --graph_def_file=tflite_graph.pb  \
--input_shapes=1,320,320,3 \
--input_arrays normalized_input_image_tensor \
--output_arrays "TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3" \
--mean_values 128 \
--std_dev_values 128 \
--allow_custom_ops \
--change_concat_input_ranges=false \
--allow_nudging_weights_to_use_fast_gemm_kernel=true \
--inference_type=QUANTIZED_UINT8

Then compile to edgetpu:

edgetpu_compiler -s out.tflite

Then it should works with our default object_detection demo!

Here is the link to download the model!

[DLMasterCat]

@Namburger

Does version of tflite_convert affect the quantization?
I need do this

--output_file=out.tflite --graph_def_file=tflite_graph.pb  --input_shapes=1,320,320,3 --input_arrays normalized_input_image_tensor --output_arrays "TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3" --mean_values 128 --std_dev_values 128 --allow_custom_ops --change_concat_input_ranges=false --allow_nudging_weights_to_use_fast_gemm_kernel=true --inference_type=QUANTIZED_UINT8 --experimental_new_converter=false --post_training_quantize --enable_v1_converter

then I could convert it.
However, every node inside the model still is float32

[sinclairnick]

Thanks @Namburger, I'm definitely still interested. I'll have a look at doing this soon and let you know how I go!

[Namburger]

@sinclairnick It got 2x speedup on the edgetpu as mentioned in this paper: https://arxiv.org/abs/2004.14525 🗡️
@DLMasterCat I don't believe so, but if you donwload the model from here there should be a float32 model directory and a uint8 model directory, using the uint8 directory should not yields any float tensors except for the post processing ops

[parthjdoshi]

@Namburger I used the model from here.

Here is the link to download the model!

However, I am facing a number of issues. There's multiple detections for the same person, indicating NMS isn't working. Secondly, the detections in itself are very inaccurate. Other objects are constantly identified as the Person class.

Can you please help me with this?

[Namburger]

@Arsenal-2004 sorry, not sure if I can help more with this since this model is owned by tensorflow, however, I've been using it without issue and have made a tutorial on it here: https://github.com/Namburger/edgetpu-ssdlite-mobiledet-retrain
Maybe it'll give you more insignt?

[parthjdoshi]

Yeah, I did follow the tutorial, but I was still facing these issues after that.