Videos: Official Nvidia transcoding suppport

[maximecurioni]

What does not work as described in the documentation?

Up to the last 2022-01-21 release, it was possible to specify the h264_nvenc encoder to ffmpeg for GPU-based transcoding with Nvidia cards.

The latest 2022-03-02 release no longer offers a version of ffmpeg with that capability.

How can we reproduce it?

Here are the verification steps in 2022-03-02:

root@photoprism:/photoprism-storage/config# docker-compose exec photoprism bash
root@5846c5714486:/photoprism# uname -a
Linux 5846c5714486 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux
root@5846c5714486:/photoprism# cat /etc/debian_version
11.2
root@5846c5714486:/photoprism# ffmpeg -loglevel quiet -encoders | grep h264_
 V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)
 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
 V..... h264_vaapi           H.264/AVC (VAAPI) (codec h264)

Notice how h264_nvenc is not part of that list.

What behavior do you expect?

Here are the exact same steps in 2022-01-21:

root@photoprism:/photoprism-storage/config# docker-compose exec photoprism bash
root@bace6e400f6e:/photoprism# uname -a
Linux bace6e400f6e 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 x86_64 x86_64 GNU/Linux      
root@bace6e400f6e:/photoprism# cat /etc/debian_version
11.0
root@bace6e400f6e:/photoprism# ffmpeg -loglevel quiet -encoders | grep h264_
 V....D h264_nvenc           NVIDIA NVENC H.264 encoder (codec h264)
 V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)
 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
 V....D h264_vaapi           H.264/AVC (VAAPI) (codec h264)

What could be the cause of your problem?

The ffmpeg version packaged in the 2022-03-02 was not compiled with support for h264_nvenc encoder.

Can you provide us with example files for testing, error logs, or screenshots?

Available to provide more information if needed

Which software versions do you use?

Available to provide more information if needed

On what kind of device is PhotoPrism installed?

Available to provide more information if needed

Do you use a Reverse Proxy, Firewall, VPN, or CDN?
None

[maximecurioni]

Here is the respective version information of ffmpeg.

2022-01-21

root@db0717d57a18:/photoprism# ffmpeg -version
ffmpeg version 4.4-6ubuntu5 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-7ubuntu1)
configuration: --prefix=/usr --extra-version=6ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100

2022-03-02

root@5846c5714486:/photoprism# ffmpeg -version
ffmpeg version 4.3.3-0+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10 (Debian 10.2.1-6)
configuration: --prefix=/usr --extra-version=0+deb11u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 51.100 / 56. 51.100
libavcodec     58. 91.100 / 58. 91.100
libavformat    58. 45.100 / 58. 45.100
libavdevice    58. 10.100 / 58. 10.100
libavfilter     7. 85.100 /  7. 85.100
libavresample   4.  0.  0 /  4.  0.  0
libswscale      5.  7.100 /  5.  7.100
libswresample   3.  7.100 /  3.  7.100
libpostproc    55.  7.100 / 55.  7.100

[maximecurioni]

For anyone who wants a temporary fix, the testing version of ffmpeg works as expected.

Here is how to add it to a custom Dockerfile:

FROM photoprism/photoprism:latest

RUN echo "\ndeb http://ftp.us.debian.org/debian testing main" > /etc/apt/sources.list
RUN apt-get update
RUN apt-get install -t testing ffmpeg

Here is the final result:

root@f366e79769af:/photoprism# ffmpeg -version
ffmpeg version 4.4.1-3 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Debian 11.2.0-14)
configuration: --prefix=/usr --extra-version=3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100
root@f366e79769af:/photoprism# ffmpeg -loglevel quiet -encoders | grep h264_
 V....D h264_nvenc           NVIDIA NVENC H.264 encoder (codec h264)
 V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)
 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
 V....D h264_vaapi           H.264/AVC (VAAPI) (codec h264)

[lastzero]

Our default image now ships with the version in that comes with Debian 11: https://packages.debian.org/

However, We still provide an Ubuntu based image, check Docker Hub: https://hub.docker.com/r/photoprism/photoprism/tags?page=1&name=impish

Did you follow the instructions in our Docker Compose example?

https://dl.photoprism.app/docker/docker-compose.yml

In particular, you need to uncomment this and restart:

PHOTOPRISM_INIT: "gpu tensorflow"

Also make sure to share all necessary devices, for example:

devices:
    - "/dev/nvidia0:/dev/nvidia0"
    - "/dev/nvidiactl:/dev/nvidiactl"

You can also manually install packages in the container as needed.

[lastzero]

Note we never had official Nvidia support, so I don't consider this a bug.

[lastzero]

The nvidia-vdpau-driver for hardware video acceleration is installed here if you selected the "gpu" option in PHOTOPRISM_INIT:

https://github.com/photoprism/photoprism/blob/develop/scripts/dist/install-gpu.sh#L36

[maximecurioni]

Thank you @lastzero. Sorry for misidentifying this as a bug.

I'll give PHOTOPRISM_INIT a try. Feel free to close this issue if it's not applicable to the project.

As always, much appreciated for the quick response.

[dailow]

For anyone who wants a temporary fix, the testing version of ffmpeg works as expected.

Here is how to add it to a custom Dockerfile:

FROM photoprism/photoprism:latest

RUN echo "\ndeb http://ftp.us.debian.org/debian testing main" > /etc/apt/sources.list
RUN apt-get update
RUN apt-get install -t testing ffmpeg

Here is the final result:

root@f366e79769af:/photoprism# ffmpeg -version
ffmpeg version 4.4.1-3 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Debian 11.2.0-14)
configuration: --prefix=/usr --extra-version=3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100
root@f366e79769af:/photoprism# ffmpeg -loglevel quiet -encoders | grep h264_
 V....D h264_nvenc           NVIDIA NVENC H.264 encoder (codec h264)
 V..... h264_omx             OpenMAX IL H.264 video encoder (codec h264)
 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
 V....D h264_vaapi           H.264/AVC (VAAPI) (codec h264)

Thanks for this. Installing from the testing repo includes the required code for ffmpeg to work. I tried using gpu for PHOTOPRISM_INIT, but while that did install the nvidia libraries it did not have the version of ffmpeg with nvenc compiled.