Running on GPU/TPU

[niccolopetti]

Hi, thanks for the effort you have put into this project,
I've been using the script but it takes the cpu usage to 100% and takes quite some time to process videos lasting a couple of hours,
Are there any chances we could use our GPUs/TPUs to improve the performace?

[morrolinux]

Hi, I think Tensorflow should use GPU by default if you have an Nvidia card with CUDA drivers, but I'm not 100% sure, I'd have to look into it. It also depend on which part is slower for you, can you paste the runlog? Il lun 18 gen 2021, 14:46 Niccolò Petti <notifications@github.com> ha scritto:

…

Hi, thanks for the effort you have put into this project, I've been using the script but it takes the cpu usage to 100% and takes quite some time to process videos lasting a couple of hours, Are there any chances we could use our GPUs/TPUs to improve the performace? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#11>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE22U3V5EB5U3BIK2LN2X6TS2Q3TXANCNFSM4WHKP4GQ> .

[niccolopetti]

Yes I have a GTX1050 with cuda and cudnn on ubuntu 20.10,
the slowest part is in the Cut and Merge function, that can take the CPU usage to 100% for long periods of time

niccolo@VivobookPro:/media/niccolo/DATA/simple-ehm$ python3 simple_ehm-runnable.py video.mp4 
analyzing track...
100%|█████████████████████████▉| 32968000/33000000 [00:25<00:00, 1300154.62it/s]CUT and MERGE: running 6 ffmpeg simultaneous instances.
33008000it [00:40, 1300154.62it/s]                                              
 11%|████▉                                      | 10/88 [00:20<04:34,  3.51s/it]
 12%|█████▍                                     | 11/88 [00:24<04:37,  3.60s/it]

 25%|██████████▊                                | 22/88 [01:36<05:01,  4.57s/it

[morrolinux]

So the issue is not tensorflow but rather ffmpeg being slow to encode on the CPU. Yes we can GPU-accelerate that, it's doable, but different APIs are provided for different cards (Intel/AMD/Nvidia) and making a universal kind of solution like an --enable-gpu flag on launch would need a lot of checks and modification to the current pipeline, perhaps even better by using ffmpeg python API instead of opening Popen processes.
It's a lot of boring work which I'd be willing to do if I personally needed that and I had the spare time to.
But I'm not so sure right now I can promise you that. Anyways PRs are always welcome

[niccolopetti]

Thanks for the answer, I am glad to know that it is doable and once I'll have more spare time to dedicate to it I'll try to do it and make a Pr if I succeed,
any help from the community is welcome