Inference Time

[rozental]

Hi,

I used the VOC2007 example, and I got an average test time of 0.2 sec per image, while using the original Caffe code on the same computer took 0.016 sec per image. Both test were on VGG architecture.

Another difference I've found was that the TF implementation takes much more memory. I could use tests on batch of size <=2. Otherwise my server ran out of memory. While on Caffe I could use bigger batches.

The time tests were made using the eval_ssd_network.py file.

• Is there a better way to do it?
• Does it seem OK for it to be much slower and heavier than the Caffe implementation?

Thanks

[balancap]

Thanks for your feedback.
For inference time benchmark, the Caffe implementation is purely C++, so clearly better optimized. If you want to compare, it could be better to use the SSD notebook with numpy post-processing: the problem with the test script is that it is purely TF-based, in order to be able to record everything for TensorBoard. But TF tends to be slower than pure Numpy implementation for the post-processing pipeline, which is loop based.

For the memory aspect, I think it is unfortunately one of TF drawback, it uses more memory than some other more optimized framework. I guess it is part of the trade-off to get more flexibility. I'll try to see if I can improve a bit on that aspect by moving more elements of the pipeline to the CPU.