Using GPU for object detection during testing time.

[phongnhhn92]

Hello guys, I successfully trained my object detector using my custom dataset.Using the file object_detection_tutorial.ipynb provided from the original object detection module, I observed the fact that during the inference time, the CPU has to work a lot (almost 100%) while the GPU usage doesnt change much.
(boxes, scores, classes, num_detections) = sess.run([boxes, scores, classes, num_detections], feed_dict={image_tensor: image_np_expanded})
I suspect that 'feed_dict={image_tensor: image_np_expanded})` will use CPU for detection so I am asking you guys that is there any ways to utilize GPU for object detection using Tensorflow ?
In my case, it took 0.3 s for SSD_MobileNets but FasterRCNN_resnet 101 took almost 8 second to make prediction for one image.

[reedwm]

/CC @tombstone

[vvsotnikov]

You should install Tensorflow with GPU support.

[phongnhhn92]

You mean I should install tensorflow from source ? I mean i doubt that would make the difference since I installed tensorflow-gpu using pip and all driver cuda cudnn recoginzed.

[gustavz]

Is there any news concerning this issue? I am also experiencing very low GPU Usage while inference

[phongnhhn92]

I am still waiting for some fixes ? May be loading image into numpy format is the problem .

[joskaaaa]

@phongnhhn92 did you ever resolve your issue, or find an alternative to feed_dict for prediction? Using feed_dict with numpy arrays is very inefficient when applying prediction to multiple images, and causes OOM errors with too many image examples.

[tensorflowbutler]

Hi There,
We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing.
If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.