Support for TensorFlow (python) inference should be added

[jmplaza]

So DetectionSuite, which is written in C++, may invoke inference of a neural network from TensorFlow-python for detection.

Maybe some pre-processing of the images can be required before injecting them into the TensorFlow network.

Maybe some post-processing of the network results (bounding boxes...) can be required to deliver them in the right structure to compute statistics.

[vinay0410]

Yes sir,
Preprocessing is required since tensorflow returns normalized bounding boxes and have to be converted so as to be used by DetectionSuite.

Similarly, the image had to be converted to C++ PyObject PyArray.
And, I have already implemented both of them here in TensorFlowInferencer.cpp.
This file handles both pre-processing and post-processing.

But, I haven't updated the documentation to display tensorflow support.
And was thinking of updating GitHub wiki for the same.

Is that right?
Or should I update documentation somewhere else ?

Thanks

[jmplaza]

Hi @vinay0410, great.

The DetectionSuite's github is the reference place for documentation, and also the wiki at the JdeRobot web page. Please feel free to update both, do not feel embarrassed.

Current documentation is kind of minimal, but it will grow and we will improve it in the following months, just to make easy to other developers to use this tool and replicate our (future) tutorials.

[naxvm]

Hello Vinay! I have worked on the ObjectDetector output. The network (both TF and Keras versions) "returns" on each prediction the bounding boxes already scaled to the input image size, I already handled that. In addition, it returns the class(directly the string, instead of an index) and the corresponding score.

I said "returns" because it is not the return of a call. As the network is an object (DetectionNetwork class), these outputs are callable properties (network.boxes; network.detections; network.scores). This has thought made on a way that allows you to only set the input and then retrieve the output (the network must be running continuously on its pertinent thread). Have a look on the component itself!

Feel free to contact me for anything, or any kind of help.

[vinay0410]

Hi @naxvm,
Thanks for the support!
Though TensorFlow support has already been added. I am using C++ to call python functions for running inference which returns bounding boxes, classes, confidence scores, etc just as you mentioned.
Checkout this pull request #19.

I have also made a video testing the same in DetectionSuite, metioned below:
Link to Video

It uses frozen inference graphs, just like dl-objectdetector to load the model and run inferences. The config file isn't necessary because of the same reason, and any file can be selected, just as depicted in the video.

You can try running an inference, and please do share your valuable suggestions.

Only, thing left for this issue, is to update the documentation regarding tensorflow support.

Thanks!

[vinay0410]

Here’s the pull request updating documentation and minor improvements for the inferencer.
I have updated Github’s wiki extensively, but since github has no option for pull request for a wiki, I have updated my local clone and mentioned it’s link.
Please have a look !
Link to my Local clone of Github Wiki

[naxvm]

Hello Vinay,

Your TF implementation seems fantastic for me, as we only need the frozen graph file, as you mention. The video inference seems promising, I was wondering why it takes that much time to make the prediction on the image. Is it because of the Python/C++ conversion? It should take far less time.

Regards!

[vinay0410]

Hi @naxvm ,
Actually,
It was using Faster R-CNN on a CPU which very slow, therefore, each inference was taking approximately 25 sec.
I will make video using SSD MobileNet which 30x faster than Faster R-CNN on CPU and post it.

Though, I will update my python file for major speed improvements in a couple of days.

Thanks for the feedback!