Batch Inference

[filipetrocadoferreira]

How to easily take advantage of batch processing during inference?

[kampelmuehler]

@filipetrocadoferreira I've implemented this for Faster RCNN and so far it seems to work ok with TEST.IMS_PER_BATCH > 1. It's not a whole lot of work to do this yourself.

[filipetrocadoferreira]

Which files did you change?

[kampelmuehler]

All changes are to be made in lib/core/test.py and lib/core/test_engine.py. (+ adding a TEST.IMS_PER_BATCH config option)

[filipetrocadoferreira]

Nice, feel free to post the commit 👍

[kampelmuehler]

Sure, there you go:
kampelmuehler/Detectron@2beae3e
note that it's neither a clean nor a comprehensive implementation.

[filipetrocadoferreira]

Thanks a lot :)
Do you think it needs changes for Mask/Keypoint prediction?

[kampelmuehler]

Yes it will for sure. See whereever it says Not implemented, or better compare with the non _batch methods.
You'll get the idea.

[filipetrocadoferreira]

Keypoint Batch Inference is not straight forward. I'm facing some problems here:

_get_rois_blob(boxes, im_scales)

worst. Now I'm aware that single image inference is composed by multiple data exchange between gpu<->cpu which will be ,prolly, the main bottleneck of performance