Tensorflow Object Detection API using Faster RCNN on Android devices.

[AliceDinh]

Please go to Stack Overflow for help and support:

http://stackoverflow.com/questions/tagged/tensorflow

Also, please understand that many of the models included in this repository are experimental and research-style code. If you open a GitHub issue, here is our policy:

1. It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
2. The form below must be filled out.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

---

System information

• What is the top-level directory of the model you are using: object_detection
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Window 10
• TensorFlow installed from (source or binary): binary, installed from pip: pip install tensorflow
• TensorFlow version (use command below): 1.8
• Bazel version (if compiling from source): None
• CUDA/cuDNN version: None
• GPU model and memory: None
• Exact command to reproduce: None

You can collect some of this information using our environment capture script:

https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

You can obtain the TensorFlow version with

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Describe the problem

I used TensorFlow Android Camera Demo, which is TF Detect, at this link:
(https://github.com/tensorflow/tensorflow/tree/r1.9/tensorflow/examples/android)
I trained my own model using SSD w/Mobilenet detector using my own data, then I imported the frozen model (about 20MB) to Android project and it was working well on my phone: Samsung S7Edge, 4GB Ram.
Now, I changed from SSD w/Mobilenet detector to Faster RCNN because Faster RCNN gives higher accuracy as many people stated. The frozen model is checked on Jupyter Notebook and the result is good. The frozen model is about 110MB and I imported to Android project. With 4GB Ram of Samsung S7Edge I believe it will be okies but there is no output result and the app crashed. So, my questions are:

Is Faster RCNN model not suitable for mobile? what should I do to make it work on Mobile?

Source code / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

[karmel]

@derekjchow - can you comment on device compatibility for Faster RCNN?

[derekjchow]

Faster-RCNN models tend to be very compute intensive (compared to SSD models). Additionally, Faster-RCNN models contain a crop-and-resize step that is not supported in the TFLite run time. For these reasons, we would not recommend using them for mobile inference.

We highly recommend using SSD models for mobile. In particular, we've done a lot of work to support 8-bit quantized inference for SSD models in TFLite. In addition, if you use TF-Lite, certain phones will support hardware acceleration via NNAPI.

Could you describe which SSD models you're working with? It's quite likely that you can get a more accurate SSD model. For example, you can try changing the feature extractor or input resolution. See our paper Speed/accuracy trade-offs for modern convolutional object detectors for more details. In addition, we have also released FPN/PPN SSD networks which may also give additional performance.

[AliceDinh]

@derekjchow Thanks for your reply.
I am using SSD (ssd_mobilenet_v1_coco.config) to train my own model: detect numbers on ATM card (like SamsungPay app). The ATM card has the standard size 660x416 and digits on cards can be considered medium/small objects that challenging SSD detector.

If I use default SSD configuration, inputs are 660x416 images, annotations are made based on those inputs, after 18K steps, the loss is about 1.2 and the accuracy is low.

I reckoned that due to the resizing step of SSD to 300x300, my inputs' size 660x416 (width:height !=1:1) made the training time longer and the accuracy lower, so I changed my inputs to 660x660, re-annotated, re-trained. After all, the accuracy is not improved.

So, please recommend me what should I do to improve the accuracy in my case!

[justinshaohi]

@AliceDinh du you has find some idea to improve?

[AliceDinh]

@justinshaohi I am trying different sizes of input, low resolution, then high resolution, that's all I can do at the moment.

[justinshaohi]

@AliceDinh i'm trying faster-rcnn-lowproposals that has high accuary now,it cost 30s in huawei mate10, what cost time you want be?

[AliceDinh]

@justinshaohi I think <10s is acceptible. Which one did you use?

faster_rcnn_nas_lowproposals_coco 
faster_rcnn_resnet50_lowproposals_coco
faster_rcnn_resnet101_lowproposals_coco
faster_rcnn_inception_resnet_v2_atrous_lowproposals_coco

[justinshaohi]

@AliceDinh <10s you can try ssd_resnet_fpn,about 8s. i use faster_rcnn_inception_resnet_v2_lowproposals

[AliceDinh]

@justinshaohi

1. Which config file I need for ssd-resnet-fpn and faster-rcnn-inception-resnet-v2-lowproposals, from this:
   https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs
2. What's about the accuracy? my model cannot detect all 16 digits in cards

[justinshaohi]

@AliceDinh ssd_fpn ,faster_rcnn,you should modify proposal

[AliceDinh]

@justinshaohi thanks, I'll let you know the result