Transcoding - support multi-gpu and nvenc load balance

[fcqpl]

Hello,
Is OME supports multi-gpu in hardware transcoding?
Maybe it's worth considering add support for assign GPU based on GPU load to transcode stream?

[Keukhan]

Hi. @fcqpl

Thank you for a good idea.

Multi-GPU is not currently supported. We will make a plan to support multi-GPU as soon as possible.

Thanks.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mpisat]

from my experience, distributing load to multiple GPUs (even to the 2 GPUs of same brand, same model) is difficult.

I mean...

nvidia-smi -q | grep -A4 Utilization
    Utilization
        Gpu                               : 48 %
        Memory                            : 4 %
        Encoder                           : 20 %
        Decoder                           : 85 %
--
    Utilization
        Gpu                               : 7 %
        Memory                            : 1 %
        Encoder                           : 8 %
        Decoder                           : 0 %

30 streams with 1080p input, round robin to ffmpeg using -gpu %i (0 or 1) along with other parameters, nope, nada.
Why it has to be so difficult? :)

[mpisat]

Actually, I think I figured it out.
after a long day with trial and error, and chatgpt

if ovenmedia can launch ffmpeg processes like these

root      808101 41.3  0.2 11594092 413624 pts/6 SLl+ 13:43   1:47 ffmpeg -re -i bbb_1080p_5mbps_aac.mp4 -async 1 -vsync -1 -filter_complex [0:v]scale=w=-1:h=720[720p] -map [720p] -c:v h264_nvenc -gpu 0 -f flv -metadata streamname=stream_54_720p rtmp://localhost:19350/live/stream_54_720p
root      808206 43.0  0.2 11593892 415688 pts/6 SLl+ 13:43   1:50 ffmpeg -re -i bbb_1080p_5mbps_aac.mp4 -async 1 -vsync -1 -filter_complex [0:v]scale=w=-1:h=720[720p] -map [720p] -c:v h264_nvenc -gpu 1 -f flv -metadata streamname=stream_55_720p rtmp://localhost:19350/live/stream_55_720p
root      808308 42.9  0.2 11594040 415752 pts/6 SLl+ 13:43   1:50 ffmpeg -re -i bbb_1080p_5mbps_aac.mp4 -async 1 -vsync -1 -filter_complex [0:v]scale=w=-1:h=720[720p] -map [720p] -c:v h264_nvenc -gpu 0 -f flv -metadata streamname=stream_56_720p rtmp://localhost:19350/live/stream_56_720p
root      808409 40.8  0.2 11593960 413552 pts/6 SLl+ 13:43   1:44 ffmpeg -re -i bbb_1080p_5mbps_aac.mp4 -async 1 -vsync -1 -filter_complex [0:v]scale=w=-1:h=720[720p] -map [720p] -c:v h264_nvenc -gpu 1 -f flv -metadata streamname=stream_57_720p rtmp://localhost:19350/live/stream_57_720p
root      808510 43.3  0.2 11593852 420076 pts/6 SLl+ 13:43   1:50 ffmpeg -re -i bbb_1080p_5mbps_aac.mp4 -async 1 -vsync -1 -filter_complex [0:v]scale=w=-1:h=720[720p] -map [720p] -c:v h264_nvenc -gpu 0 -f flv -metadata streamname=stream_58_720p rtmp://localhost:19350/live/stream_58_720p
root      808611 41.7  0.2 11593916 421664 pts/6 RLl+ 13:43   1:46 ffmpeg -re -i bbb_1080p_5mbps_aac.mp4 -async 1 -vsync -1 -filter_complex [0:v]scale=w=-1:h=720[720p] -map [720p] -c:v h264_nvenc -gpu 1 -f flv -metadata streamname=stream_59_720p rtmp://localhost:19350/live/stream_59_720p

it can utilize both GPU equally. However, I'm not using GPU decoding in the example above. the GPU I have only has one decoder and three encoder chips and I wanted to see how CPU decoding / GPU encoding would work. Well, it worked wonderfully.

this is the standard ffmpeg from ubuntu 22.04

# ffmpeg -v
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100

[mpisat]

what's difficult is to use gpu decoding and scaling in multiple gpu setup. If you can't specify GPU, then it goes to first gpu always.
the solution I used was
export CUDA_VISIBLE_DEVICES=1 # for GPU ID 1
in the process I call ffmpeg
and since I did round robin, when it was GPU ID 0
export CUDA_VISIBLE_DEVICES=0
for the other process in round robin format.
this allowed me to do load balance in the gpu.

root      968174 33.4  0.2 6701016 562228 pts/9  Dl+  00:14   0:16 ffmpeg6 -init_hw_device cuda=my_gpu0 -filter_hw_device my_gpu0 -c:v h264_cuvid -re -stream_loop -1 -i bbb_1080p_5mbps_aac.mp4 -async 1 -vsync -1 -filter_complex [0:v]hwupload_cuda,scale_cuda=w=-1:h=720[720p];[0:v]hwupload_cuda,scale_cuda=w=-1:h=540[540p];[0:v]hwupload_cuda,scale_cuda=w=-1:h=360[360p] -map [720p] -c:v h264_nvenc -preset p1 -b:v 2500k -maxrate 2500k -bufsize 8000k -f null /dev/null -map [540p] -c:v h264_nvenc -preset p1 -b:v 1200k -maxrate 1200k -bufsize 4000k -f null /dev/null -map [360p] -c:v h264_nvenc -preset p1 -b:v 600k -maxrate 600k -bufsize 2000k -f null /dev/null

root      968181 33.8  0.2 6700132 557200 pts/9  Dl+  00:14   0:15 ffmpeg6 -init_hw_device cuda=my_gpu1 -filter_hw_device my_gpu1 -c:v h264_cuvid -re -stream_loop -1 -i bbb_1080p_5mbps_aac.mp4 -async 1 -vsync -1 -filter_complex [0:v]hwupload_cuda,scale_cuda=w=-1:h=720[720p];[0:v]hwupload_cuda,scale_cuda=w=-1:h=540[540p];[0:v]hwupload_cuda,scale_cuda=w=-1:h=360[360p] -map [720p] -c:v h264_nvenc -preset p1 -b:v 2500k -maxrate 2500k -bufsize 8000k -f null /dev/null -map [540p] -c:v h264_nvenc -preset p1 -b:v 1200k -maxrate 1200k -bufsize 4000k -f null /dev/null -map [360p] -c:v h264_nvenc -preset p1 -b:v 600k -maxrate 600k -bufsize 2000k -f null /dev/null

and each process has it's own ENV variable

a node.js code snipped:
let ffmpegProcess = spawn(cmd[0], cmd.slice(1), {
env: {
...process.env,
CUDA_VISIBLE_DEVICES: i % gpuDevices,
},
});
this sets CUDA_VISIBLE_DEVICES: 0 or 1, and I only have 2 so it works.

this is the result

nvidia-smi -q -l 3 | grep -A4 Utilization
    Utilization
        Gpu                               : 84 %
        Memory                            : 8 %
        Encoder                           : 28 %
        Decoder                           : 83 %
--
    Utilization
        Gpu                               : 89 %
        Memory                            : 8 %
        Encoder                           : 27 %
        Decoder                           : 82 %
--
    Utilization
        Gpu                               : 84 %
        Memory                            : 8 %
        Encoder                           : 29 %
        Decoder                           : 85 %
--
    Utilization
        Gpu                               : 91 %
        Memory                            : 8 %
        Encoder                           : 29 %
        Decoder                           : 88 %

[mpisat]

by utilizing NO Copy on ffmpeg, memory utilization is much better.

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -re -i bbb_1080p_5mbps_aac.mp4 -vf scale_cuda=1280:720 -c:a copy -c:v h264_nvenc -b:v 2000k -f null /dev/null -vf scale_cuda=960:540 -c:a copy -c:v h264_nvenc -b:v 1200k -f null /dev/null -vf scale_cuda=640:360 -c:a copy -c:v h264_nvenc -b:v 550k -f null /dev/null

and with CUDA_VISIBLE_DEVICES: 0 (GPU ID) trick, it is still possible to load balance GPUs.

I can't provide MR, but I can assist @Keukhan if he needs it.

[getroot]

This has been released. @Keukhan Please introduce the features you updated.