GPU not really utilized well

[ichitaka]

So I've run a few tests, as I've noticed that PyFeat is quite slow in comparison to something like OpenFace2. Turns out, while the CPU is always utilized, the GPU is not. This seems to be, because the data is loaded 1-by-1 through OpenCV instead of using a proper GPU library for lists of images. I think it makes sense, if a list of images is given to a Detector for detect_image, to load and predict the images batch wise through torch.DataLoaders.

[TiankangXie]

Hello @ichitaka ! Sorry for the late reply, but that's definitely an excellent suggestion! Basically, we used frame-by-frame reading because at that time we don't know if you could load videos with the Pytorch data loader. But in the next (soon) release we will address this problem! Thank you so much for the insights!

[dexterdev]

@TiankangXie : Any update from your side on this? I use resmasknet model to get emotion scores data from videos. And Py-Feat is extremely slow. Any temporary solutions at least?

[TiankangXie]

@dexterdev I am sorry for the late reply. Have you tried to increase the parameter - batch_size, and increase the skip_frames? Currently, these are the two parameters that could affect the processing speed. Increasing the batch_size and skip_frames should reduce the processing time proportionally.

We are reading the frames of the video by frames with the OpenCV cap.read function. This is not optimal and we are working on a major update soon!
Again thank you for the feedbacks!

[dexterdev]

No I havent tried those options. And also I was always trying to load all frames. Thank you. Looking forward for the updated Py-Feat :)

[ljchang]

Thanks @ichitaka for the excellent suggestion. We have completely refactored our toolbox to use the pytorch data loaders. We can report that there is now dramatic speedups on CPU, but also the ability to use GPUs as well. We are finishing up testing and will be updating our release very shortly (#133).

[ichitaka]

Great update!

[dexterdev]

Great. I think this is at right time for our requirements. Thanks for the great package and recent upgrade <3

[ljchang]

Hopefully this is now addressed in version 0.5.0 with #144