Object Detection via Faster R-CNN

[pribadihcr]

Hi @beniz ,

faster RCNN like https://github.com/rbgirshick/py-faster-rcnn is a very useful application. It's implemented and trained in Caffe framework.
Do you have a plan to implement it in deepdeteect?

Thanks,

[beniz]

@pribadihcr yes it is indeed on the roadmap, though not fast-tracked yet. It would be useful if you could share what the most important features are for you regarding faster R-CNN wrt DD's API and server side. Typically, are you mostly interested in the training phase or an in-production prediction phase (while training with py-faster-rcnn) ? Regarding the models, are you looking forward using the existing set of pre-trained ones, or building your own ?

[pribadihcr]

I think an in-production prediction phase is the first. We can train the
model using existing faster RCNN framework (e.g. py-faster-rcnn).
Regarding the models, well, I want to localize and detect spesific objects. So, I think I
need custom model.

On Mon, Jan 4, 2016 at 12:22 PM, Emmanuel Benazera <notifications@github.com

wrote:

@pribadihcr https://github.com/pribadihcr yes it is indeed on the
roadmap, though not fast-tracked yet. It would be useful if you could share
what the most important features are for you regarding faster R-CNN wrt
DD's API and server side. Typically, are you mostly interested in the
training phase or an in-production prediction phase (while training with
py-faster-rcnn) ? Regarding the models, are you looking forward using the
existing set of pre-trained ones, or building your own ?

—
Reply to this email directly or view it on GitHub
https://github.com/beniz/deepdetect/issues/43#issuecomment-168585714.

[beniz]

OK, thanks, that was my guess. There are two possible ways of doing this. First is fast-tracking of features, and we do this through sponsorship. Current running sponsorships are not about images (as can be seen from last PR) and this is typically why faster-RCNN is not in yet. Second is slower track where the prediction pipeline can be brought in but with no deadline attached. Timeline for this one track also depends on whether you plan on participating in the development.
Regarding custom modeling, you can use some of the models we keep releasing (e.g. http://www.deepdetect.com/applications/model/) from the datasets we collect and process, or use private datasets of labeled data and build your own. Of course the currently relased image models are for classification, not detection, but we would certainly release some new ones once the faster-RCNN code is in, as needed. Let me know your thoughts and whether this can fit your needs.

[beniz]

After review, py-faster-cnn is lacking some features for integration into production / commodity deep learning tools such as dd, more precisely:

• the Python ROI layer needs to be replaced by the C++ and CUDA implementations, see https://github.com/rbgirshick/caffe-fast-rcnn/blob/fast-rcnn/src/caffe/layers/roi_pooling_layer.cu and Add ROIPoolingLayer from Fast-RCNN for Detecition & Attention BVLC/caffe#2670
• the prediction step does not support ROI pooling over batches of images, as far as I understand from the code and im_detect function, and this is no straightforward task, see ROIPooling layer in fast and faster R-CNN Lasagne/Lasagne#565

These are just a few of the many details to consider in order to turn this into a commodity.

[zyyang]

FYI: caffe with py-faster-rcnn has been updated from original git.

[vuvko]

Hi, I'm helping with seminars on deep learning and one of them is about RCNN.
Thought it can be usefull, here's theano's implementation of RoIPooling (only gpu though): https://github.com/ddtm/theano-roi-pooling

[beniz]

FTR, BVLC/caffe#4163 seems to be coming to Caffe. Once it is stabilized, we may merge it to our own modified version of Caffe, independently of whether it makes it to official Caffe. From there, there'll be a path for service integration into DD.

[beniz]

Object detection now implemented via SSD, see PR #213. Closing.