Videos: Improve Nvidia GPU Support #2613

lastzero · 2022-08-13T20:05:21Z

There may be better ways to enable support for Nvidia GPUs than installing the distribution's default drivers, since Nvidia offers a Docker runtime/integration:

Videos: Official Nvidia transcoding suppport #2125 (comment)

Acceptance Criteria:

Validate the approach: Does it really work? Does it reduce the CPU load?
Document compatibility constraints: What hardware/software will work? What will not work? Windows? Linux? macOS?
Document all the steps required so that other contributors and our team can get transcoding to work as expected
Create an example Docker config that we can share at https://dl.photoprism.app/docker/

mlapaglia · 2022-08-22T17:49:17Z

Validate the approach: Does it really work? Does it reduce the CPU load?

This requires the nvidia drivers to be installed on the docker host. The drivers include all of the libraries needed for decoding and encoding, tensorflow, etc.

I tested performance by removing PHOTOPRISM_INIT from the docker arguments and add the following arguments

docker run -d
-e 'NVIDIA_VISIBLE_DEVICES'='all'
-e 'NVIDIA_DRIVER_CAPABILITIES'='all'
-e 'PHOTOPRISM_FFMPEG_ENCODER'='nvidia'
--runtime=nvidia 
'photoprism/photoprism'

Once running I executed a transcode manually from the console. I took this command from the logs when I clicked on the video through the UI.
/usr/bin/ffmpeg -i /photoprism/originals/2022/08/20220819_223142_5D930210.mp4 -pix_fmt yuv420p -c:v h264_nvenc -c:a aac -preset 15 -pixel_format yuv420p -gpu any -vf format=yuv420p -rc:v constqp -cq 0 -tune 2 -r 30 -b:v 50M -profile:v 1 -coder:v 1 -f mp4 -y /photoprism/storage/sidecar/2022/08/20220819_223142_5D930210.mp4.avc

Notice transcoding speed at 1.05x. It appears that ffmpeg doesn't try to use the GPU at all.

Adding -hwaccel auto to the arguments /usr/bin/ffmpeg -hwaccel auto -i /photoprism/originals/2022/08/20220819_223142_5D930210.mp4 -pix_fmt yuv420p -c:v h264_nvenc -c:a aac -preset 15 -pixel_format yuv420p -gpu any -vf format=yuv420p -rc:v constqp -cq 0 -tune 2 -r 30 -b:v 50M -profile:v 1 -coder:v 1 -f mp4 -y /photoprism/storage/sidecar/2022/08/20220819_223142_5D930210.mp4.avc

Notice transcoding speed 1.98x and significantly lower CPU usage. Also, nvidia-smi shows an active transcode running:

graciousgrey · 2022-08-25T14:13:09Z

Thanks for testing ❤️

LordNex · 2022-09-12T02:11:11Z

Question. Will this run natively on a Jetson Nano 4gig?

Honestly I'm trying to be able to use it with Home Assistant, DoubleTake, and Frigate. Using PhotoPrism as the data source for the faces I know for training.

I have requests with those parties to hopefully tie into here based on url searches as well.

My whole approach is to have a video doorbell that will recognize anyone that I have taken a picture of and labeled and trigger an automation. I have most of it down. Now it's just trying to get your data into DoubleTake

lastzero · 2022-09-12T07:45:11Z

It will run on a Jetson, but you have to compile it and manually install the dependencies incl TensorFlow libs if you don't want to use Docker (which includes all of this in a single image).

LordNex · 2022-09-12T09:17:53Z

It will run on a Jetson, but you have to compile it and manually install the dependencies incl TensorFlow libs if you don't want to use Docker (which includes all of this in a single image).

No I'd prefer to just load the DeepStack base image minus running DeepStack and then run it under docker compose. That way I know the dependencies should already be there for TensorFlow and the CUDA drivers. Should logical to you?

lastzero · 2022-09-12T10:02:14Z

You need the right TensorFlow libs inside the Docker image unless you only want hardware video transcoding.

lastzero · 2022-09-12T10:04:11Z

https://dl.photoprism.app/tensorflow/nvidia-jetson/

ngoldack · 2022-09-19T15:12:53Z

Hi, I still have issues getting this running.
I followed @mlapaglia steps, but it won't work.
I'm running Unraid with an Nvidia T600 installed.

When running nvidia-smi it shows Cuda installed:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA T600         Off  | 00000000:01:00.0 Off |                  N/A |
| 34%   37C    P0    N/A /  41W |      0MiB /  4096MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------

However when I start a transcode with /usr/bin/ffmpeg -i originals/2022/03/20220307_022629_7792822B.mp4 -pix_fmt yuv420p -c:v h264_nvenc -c:a aac -preset 15 -pixel_format yuv420p -gpu any -vf format=yuv420p -rc:v constqp -cq 0 -tune 2 -r 30 -b:v 50M -profile:v 1 -coder:v 1 -f mp4 -y /test.avc
I get the following error:

ffmpeg version 5.1-3 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-1)
  configuration: --prefix=/usr --extra-version=3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-shared
  WARNING: library configuration mismatch
  avcodec     configuration: --prefix=/usr --extra-version=3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libaribb24 --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc --enable-libsmbclient
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55e8abd09640] st: 0 edit list: 1 Missing key frame while searching for timestamp: 2000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55e8abd09640] st: 0 edit list 1 Cannot find an index entry before timestamp: 2000.
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'originals/2022/03/20220307_022629_7792822B.mp4':
  Metadata:
    major_brand     : XAVC
    minor_version   : 17440767
    compatible_brands: XAVCmp42iso2nras
    creation_time   : 2022-03-07T02:26:29.000000Z
  Duration: 00:00:27.36, start: 0.000000, bitrate: 107968 kb/s
  Stream #0:0[0x1](und): Video: hevc (Rext) (hvc1 / 0x31637668), yuv422p10le(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 97160 kb/s, 50 fps, 50 tbr, 50k tbn (default)
    Metadata:
      creation_time   : 2022-03-07T02:26:29.000000Z
      handler_name    : Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : HEVC Coding
    Side data:
      displaymatrix: rotation of -90.00 degrees
  Stream #0:1[0x2](und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, stereo, s16, 1536 kb/s (default)
    Metadata:
      creation_time   : 2022-03-07T02:26:29.000000Z
      handler_name    : Sound Media Handler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (rtmd / 0x646D7472), 4505 kb/s (default)
    Metadata:
      creation_time   : 2022-03-07T02:26:29.000000Z
      handler_name    : Timed Metadata Media Handler
      timecode        : 00:01:05:14
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (pcm_s16be (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_nvenc @ 0x55e8abd46400] Cannot load libcuda.so.1
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

Any suggestions?

Edit:
Usage of --runtime=nvidia in other docker containers, such as Jellyfin, works.

mlapaglia · 2022-09-19T15:16:11Z

do you also have the

-e 'NVIDIA_VISIBLE_DEVICES'='all'
-e 'NVIDIA_DRIVER_CAPABILITIES'='all'
-e 'PHOTOPRISM_FFMPEG_ENCODER'='nvidia'

flags set?

ngoldack · 2022-09-19T15:16:55Z

do you also have the

-e 'NVIDIA_VISIBLE_DEVICES'='all'
-e 'NVIDIA_DRIVER_CAPABILITIES'='all'
-e 'PHOTOPRISM_FFMPEG_ENCODER'='nvidia'

flags set?

Yes!

mlapaglia · 2022-09-19T15:18:41Z

ok try the command i used /usr/bin/ffmpeg -hwaccel auto -i /photoprism/originals/2022/08/20220819_223142_5D930210.mp4 -pix_fmt yuv420p -c:v h264_nvenc -c:a aac -preset 15 -pixel_format yuv420p -gpu any -vf format=yuv420p -rc:v constqp -cq 0 -tune 2 -r 30 -b:v 50M -profile:v 1 -coder:v 1 -f mp4 -y /photoprism/storage/sidecar/2022/08/20220819_223142_5D930210.mp4.avc

maybe try a different file to transcode?

ngoldack · 2022-09-19T15:25:29Z

I tried it with the same file again and with a diffrent one - even more errors:

ffmpeg version 5.1-3 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-1)
  configuration: --prefix=/usr --extra-version=3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-shared
  WARNING: library configuration mismatch
  avcodec     configuration: --prefix=/usr --extra-version=3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libaribb24 --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc --enable-libsmbclient
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55c6944c3640] st: 0 edit list: 1 Missing key frame while searching for timestamp: 1000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55c6944c3640] st: 0 edit list 1 Cannot find an index entry before timestamp: 1000.
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'originals/2022/06/20220614_210734_C2D02640.mp4':
  Metadata:
    major_brand     : XAVC
    minor_version   : 16785407
    compatible_brands: XAVCmp42iso2
    creation_time   : 2022-06-14T21:07:34.000000Z
  Duration: 00:02:16.80, start: 0.000000, bitrate: 51758 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-4, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 49585 kb/s, 50 fps, 50 tbr, 50k tbn (default)
    Metadata:
      creation_time   : 2022-06-14T21:07:34.000000Z
      handler_name    : Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
  Stream #0:1[0x2](und): Audio: pcm_s16be (twos / 0x736F7774), 48000 Hz, stereo, s16, 1536 kb/s (default)
    Metadata:
      creation_time   : 2022-06-14T21:07:34.000000Z
      handler_name    : Sound Media Handler
      vendor_id       : [0][0][0][0]
  Stream #0:2[0x3](und): Data: none (rtmd / 0x646D7472), 409 kb/s (default)
    Metadata:
      creation_time   : 2022-06-14T21:07:34.000000Z
      handler_name    : Timed Metadata Media Handler
      timecode        : 00:54:37:22
[AVHWDeviceContext @ 0x55c6944c8880] Cannot load libcuda.so.1
[AVHWDeviceContext @ 0x55c6944c8880] Could not dynamically load CUDA
Device creation failed: -1.
Device creation failed: -542398533.
[AVHWDeviceContext @ 0x55c6944e6640] Cannot open the X11 display .
Device creation failed: -1313558101.
[h264 @ 0x55c6944e70c0] Auto hwaccel disabled: no device found.
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_nvenc))
  Stream #0:1 -> #0:1 (pcm_s16be (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_nvenc @ 0x55c6944e26c0] Cannot load libcuda.so.1
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!

mlapaglia · 2022-09-19T15:46:36Z

[h264 @ 0x55c6944e70c0] Auto hwaccel disabled: no device found. are you running nvidia-smi from inside the container? it looks like it isn't seeing your gpu

jdancouga · 2022-10-06T15:27:59Z

I would like to report this method worked for me.
I am running the official photoprism docker on UnRaid (6.10.3)
Since I have multiple GPUs in the system, instead of using 'all' for 'NVIDIA_VISIBLE_DEVICES', I use the GPU UUID instead.

newness0 · 2022-10-13T12:07:07Z

Hi, I'm not an expert. can someone provide me the full docker command I need to use to take advantage of the Nvidia in containers?

wjbridge · 2022-10-24T01:42:18Z

I thought I would write up my testing so far. Hopefully, this helps. I have a single GTX 1080 GPU running a Debian server. I have my host configured using the Nvidia docker https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html using the nvidia-container-toolkit. By doing this, all that is required to enable it in the container is as follows:

  photoprism:
    image: photoprism/photoprism:latest
    container_name: photoprism
    restart: unless-stopped
    shm_size: "2gb"
    healthcheck:
      test: ["CMD-SHELL", "/opt/photoprism/bin/photoprism status | grep -i 'operational'"]
      interval: 10m
      timeout: 10s
      retries: 5
      start_period: 10s
    environment:
      TZ: $TZ
      PHOTOPRISM_EXPERIMENTAL: true
      PHOTOPRISM_INIT: tensorflow
      PHOTOPRISM_AUTH_MODE: password
      PHOTOPRISM_ADMIN_PASSWORD: **********
      PHOTOPRISM_UPLOAD_NSFW: true
      PHOTOPRISM_HTTP_COMPRESSION: gzip
      PHOTOPRISM_THUMB_FILTER: blackman
      PHOTOPRISM_LOG_LEVEL: trace
      PHOTOPRISM_READONLY: false
      PHOTOPRISM_SITE_TITLE: Photoprism
      PHOTOPRISM_FFMPEG_ENCODER: nvidia
      PHOTOPRISM_FFMPEG_BITRATE: 50
      NVIDIA_VISIBLE_DEVICES: all
      NVIDIA_DRIVER_CAPABILITIES: compute,video,utility
    networks:
      - $T2_NETWORK
    depends_on:
      - $PROXY
    security_opt:
      - seccomp:unconfined
      - apparmor:unconfined
    volumes:
      - FOLDER:/photoprism/import
      - FOLDER2:/photoprism/originals
      - FOLDER3:/photoprism/storage
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

This will enable the GPU in the container. If you execute, docker exec -it photoprism bash and run nvidia-smi, this should execute correctly without errors.

Problem `-level:v 41`

I am seeing a problem with the -level:v 41 parameter passed by default using the Nvidia encoder. If you change this parameter to level:v auto or remove it, the command works great.

Nvidia-Smi Output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.141.03   Driver Version: 470.141.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:27:00.0 Off |                  N/A |
|  0%   44C    P8     9W / 151W |      1MiB /  8117MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Error with parameter -level:v 41

[h264_nvenc @ 0x562a91baa980] InitializeEncoder failed: invalid param (8): Invalid Level.
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

I am also able to confirm that passing -hwaccel auto speeds up the transcode considerably.

Full Command:

/usr/bin/ffmpeg -hwaccel auto -i /photoprism/originals/FILENAME.mp4 -pix_fmt yuv420p -c:v h264_nvenc -c:a aac -preset 15 -pixel_format yuv420p -gpu any -vf format=yuv420p -rc:v constqp -cq 0 -tune 2 -r 30 -b:v 50M -profile:v 1 -level:v auto -coder:v 1 -f mp4 -y /photoprism/storage/sidecar/FILENAME.mp4.avc

No -hwaccel auto with no -level 41

frame=  120 fps= 27 q=27.0 size=   14080kB time=00:00:05.09 bitrate=22621.7kbits/s dup=7 drop=0 speed=1.16x

With -hwaccel auto with no -level 41

frame=  385 fps= 88 q=27.0 size=   49920kB time=00:00:13.29 bitrate=30769.3kbits/s dup=8 drop=0 speed=3.04x

Signed-off-by: Michael Mayer <michael@photoprism.app>

lastzero · 2022-11-03T15:26:23Z

@wjbridge I have updated the FFmpeg parameters based on your suggestions. Due to limited resources, this is all I can do for now. Let us know if this helps. Further improvements from our side will have to wait until the multi-user features are ready. However, we welcome any contributions!

jahanson · 2022-11-03T15:38:40Z

apt install nvidia-driver-520 installed the nvidia drivers and it paired the proper dkms package with it. I was also able to validate with nvidia-smi
I also had version 520 installed on my host.

lastzero · 2022-11-03T15:50:33Z

Based on this overview, the package would automatically install a full X11 desktop environment with it?

https://packages.ubuntu.com/jammy-updates/nvidia-driver-520

That seems a bit too much, although that probably covers everything you might need. What I didn't see is opencl and libva2, which might also be needed (I didn't check the full package dependencies, so it might be installed - I just didn't see it explicitly).

lastzero · 2022-11-03T15:52:02Z

If anyone has time to figure this out completely and test it, here is the script to update:

https://github.com/photoprism/photoprism/blob/develop/scripts/dist/install-gpu.sh

jahanson · 2022-11-03T16:26:17Z

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html

You were right about X11, my ffmpeg wont do it without an X11 session with nvidia-driver-520

I'll try the docker approach.

jahanson · 2022-11-03T18:29:47Z

Using the nvidia hook mentioned by @wjbridge and the preview docker tag I was able to get live photos to convert using my nvidia 3080.

wjbridge · 2022-11-03T23:47:00Z

I can also confirm that the preview build 221103-211eb36ea is working super! Thank you very much for making those quick changes @lastzero.

Feedback

Regarding the install-gpu.sh.
I don't know much and could be completely wrong in this opinion but I recommend doing the Nvidia docker method. This avoids the host having a different driver version than the container. It also leaves it up to the installer to make sure the correct driver is installed for that GPU. Many open-source projects rely on this method for Nvidia GPU-accelerated capability in containers. However, please disregard my ignorance if I am missing something in which this solution would not be ideal.

Logs

Thu Nov  3 19:29:23 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.85.02    Driver Version: 510.85.02    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:27:00.0 Off |                  N/A |
| 20%   58C    P2    47W / 151W |    843MiB /  8192MiB |     37%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     86848      C   /usr/bin/ffmpeg                   839MiB |
+-----------------------------------------------------------------------------+

INFO[2022-11-03T19:28:58-04:00] convert: converting 20210704_003736_3BEAA268.mp4 to 20210704_003736_3BEAA268.mp4.jpg (ffmpeg)
TRAC[2022-11-03T19:28:58-04:00] /usr/bin/ffmpeg -y -i /photoprism/originals/2021/07/20210704_003736_3BEAA268.mp4 -ss 00:00:00.001 -vframes 1 /photoprism/storage/sidecar/2021/07/20210704_003736_3BEAA268.mp4.jpg
INFO[2022-11-03T19:29:00-04:00] convert: 20210704_003736_3BEAA268.mp4.jpg created in 1.209191777s (ffmpeg)
TRAC[2022-11-03T19:29:00-04:00] metadata: found no additional sidecar file for 20210704_003736_3BEAA268.mp4
DEBU[2022-11-03T19:29:00-04:00] metadata: 20210704_003736_3BEAA268.mp4 uses utc by default (video/mp4)
INFO[2022-11-03T19:29:00-04:00] convert: ffmpeg encoder h264_nvenc selected

Signed-off-by: Michael Mayer <michael@photoprism.app>

lastzero · 2022-11-04T10:40:13Z

Added a (hopefully) ready-to-use docker-compose.yml example and updated the transcoding docs to include more details:

Let us know if it works so we can release it 👍️

Signed-off-by: Michael Mayer <michael@photoprism.app>

just5ky · 2022-11-04T12:03:32Z

yup, i can transcode using GPU now

gyto6 · 2022-11-04T15:15:43Z

As an Unraid user, I didn't suffer from any transcoding issue with my GPU so I don't expect to install the Nvidia Toolkit container.

Am I the only one in this case?

lastzero · 2022-11-04T15:31:50Z

@gyto6 The old method still works if you share the hardware devices, but you have to install the drivers manually in the Docker container. From the feedback above, the old "gpu" target for PHOTOPRISM_INIT didn`t work anymore anyway. So the fact that it is now removed should not make any difference as the result should be the same!?

gyto6 · 2022-11-04T15:49:54Z

Well, I didn't install any driver in the docker container by myself.

The host (Unraid) had the latest Stable release Nvidia drivers, 515, whereas the Photoprism container displayed the 470 drivers version, if I remembered well, with the nvidia-smi command.

I repeat, I didn't install any driver in the container, I don't know how to do it. I'd do it to test Tensorflow with the GPU if I could.

Whatever, I tested once again:

time="2022-11-04T16:38:54+01:00" level=info msg="convert: ffmpeg encoder h264_nvenc selected" time="2022-11-04T16:38:54+01:00" level=info msg="h264_nvenc: transcoding Gauthier’s iPhone/IMG_1631.MOV to avc" time="2022-11-04T16:39:06+01:00" level=info msg="h264_nvenc: created IMG_1631.MOV.avc [12.755432012s]"

So it's still working, but I don't understand as you just told that I should have installed the drivers by myself.

lastzero · 2022-11-04T15:51:32Z

Maybe FFmpeg can transcode without NVIDIA drivers being installed in the container? That seems possible, I'm really not an NVIDIA expert...

gyto6 · 2022-11-04T15:53:20Z

Maybe FFmpeg can transcode without NVIDIA drivers being installed in the container? That seems possible, I'm really not an NVIDIA expert...

Maybe indeed ! But why did the others had some troubles then. That's weird.

gyto6 · 2022-11-04T15:57:58Z

Whatever, it has always been functionnal for me, and I hope so for the others now.

Thanks for having answered my questions.

lastzero · 2022-11-04T16:03:16Z

Thanks for testing! 🎂

* merge-221105: (346 commits) UX: Improve layout of form fields in photo edit dialog Account: Disable "gender" dropdown when busy photoprism#98 Docker: Change "docker-compose" command to "docker compose" photoprism#1192 Docker: Use docker compose profiles for watchtower example photoprism#1192 Docker: Change "docker-compose" command to "docker compose" photoprism#1192 Frontend: Upgrade dependencies in package.json and package-lock.json Frontend: Update translations.json and package-lock.json Weblate: Update backend translations Weblate: Update frontend translations Weblate: Update frontend translations Develop: Add "FOR TEST AND DEVELOPMENT ONLY" warning to config files NVIDIA: Update docker-compose.yml example photoprism#2613 NVIDIA: Add docker-compose.yml example photoprism#2613 NVIDIA: Update install-gpu.sh script photoprism#2613 Frontend: Update translations.json and package-lock.json Weblate: Update frontend translations Tests: Refactor acceptance tests Tests: Add DateTime unit test Video: Change FFmpeg params for NVidia hardware transcoding photoprism#2613 Frontend: Update package-lock.json ...

doctorttt · 2023-07-01T04:54:05Z

I'm using Unraid as well but couldn't get my nvidia gpu to work. I added all the necessary variables to the container. Please help.

#3506

lastzero added help wanted Well suited for external contributors! enhancement Refactoring, improvement or maintenance task video Video Formats, Transcoding, FFmpeg, Streaming & Co labels Aug 13, 2022

lastzero mentioned this issue Aug 13, 2022

Videos: Official Nvidia transcoding suppport #2125

Closed

lastzero added a commit that referenced this issue Nov 3, 2022

Video: Change FFmpeg params for NVidia hardware transcoding #2613

06744f3

Signed-off-by: Michael Mayer <michael@photoprism.app>

lastzero added a commit that referenced this issue Nov 4, 2022

NVIDIA: Update install-gpu.sh script #2613

6d86515

Signed-off-by: Michael Mayer <michael@photoprism.app>

lastzero self-assigned this Nov 4, 2022

lastzero added a commit that referenced this issue Nov 4, 2022

NVIDIA: Add docker-compose.yml example #2613

251a69b

Signed-off-by: Michael Mayer <michael@photoprism.app>

lastzero added the please-test Ready for acceptance test label Nov 4, 2022

lastzero added a commit that referenced this issue Nov 4, 2022

NVIDIA: Update docker-compose.yml example #2613

72386be

Signed-off-by: Michael Mayer <michael@photoprism.app>

lastzero added member-feature Available to members and contributors and removed help wanted Well suited for external contributors! labels Nov 4, 2022

lastzero added released Available in the stable release and removed please-test Ready for acceptance test labels Nov 4, 2022

lastzero closed this as completed Nov 4, 2022

lastzero added plus-feature Available to PhotoPrism+ Members and removed member-feature Available to members and contributors labels Feb 1, 2023

graciousgrey removed the plus-feature Available to PhotoPrism+ Members label Jun 7, 2023

lastzero added this to Roadmap 🚀✨ Jun 8, 2023

lastzero moved this to Released 🌈 in Roadmap 🚀✨ Jun 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Videos: Improve Nvidia GPU Support #2613

Videos: Improve Nvidia GPU Support #2613

lastzero commented Aug 13, 2022 •

edited

Loading

mlapaglia commented Aug 22, 2022

graciousgrey commented Aug 25, 2022

LordNex commented Sep 12, 2022

lastzero commented Sep 12, 2022

LordNex commented Sep 12, 2022

lastzero commented Sep 12, 2022

lastzero commented Sep 12, 2022

ngoldack commented Sep 19, 2022 •

edited

Loading

mlapaglia commented Sep 19, 2022

ngoldack commented Sep 19, 2022

mlapaglia commented Sep 19, 2022 •

edited

Loading

ngoldack commented Sep 19, 2022

mlapaglia commented Sep 19, 2022

jdancouga commented Oct 6, 2022

newness0 commented Oct 13, 2022

wjbridge commented Oct 24, 2022

lastzero commented Nov 3, 2022

jahanson commented Nov 3, 2022 •

edited

Loading

lastzero commented Nov 3, 2022

lastzero commented Nov 3, 2022

jahanson commented Nov 3, 2022

jahanson commented Nov 3, 2022 •

edited

Loading

wjbridge commented Nov 3, 2022

lastzero commented Nov 4, 2022

just5ky commented Nov 4, 2022

gyto6 commented Nov 4, 2022 •

edited

Loading

lastzero commented Nov 4, 2022

gyto6 commented Nov 4, 2022 •

edited

Loading

lastzero commented Nov 4, 2022

gyto6 commented Nov 4, 2022

gyto6 commented Nov 4, 2022

lastzero commented Nov 4, 2022

doctorttt commented Jul 1, 2023

Videos: Improve Nvidia GPU Support #2613

Videos: Improve Nvidia GPU Support #2613

Comments

lastzero commented Aug 13, 2022 • edited Loading

mlapaglia commented Aug 22, 2022

Validate the approach: Does it really work? Does it reduce the CPU load?

graciousgrey commented Aug 25, 2022

LordNex commented Sep 12, 2022

lastzero commented Sep 12, 2022

LordNex commented Sep 12, 2022

lastzero commented Sep 12, 2022

lastzero commented Sep 12, 2022

ngoldack commented Sep 19, 2022 • edited Loading

mlapaglia commented Sep 19, 2022

ngoldack commented Sep 19, 2022

mlapaglia commented Sep 19, 2022 • edited Loading

ngoldack commented Sep 19, 2022

mlapaglia commented Sep 19, 2022

jdancouga commented Oct 6, 2022

newness0 commented Oct 13, 2022

wjbridge commented Oct 24, 2022

Problem -level:v 41

lastzero commented Nov 3, 2022

jahanson commented Nov 3, 2022 • edited Loading

lastzero commented Nov 3, 2022

lastzero commented Nov 3, 2022

jahanson commented Nov 3, 2022

jahanson commented Nov 3, 2022 • edited Loading

wjbridge commented Nov 3, 2022

Feedback

Logs

lastzero commented Nov 4, 2022

just5ky commented Nov 4, 2022

gyto6 commented Nov 4, 2022 • edited Loading

lastzero commented Nov 4, 2022

gyto6 commented Nov 4, 2022 • edited Loading

lastzero commented Nov 4, 2022

gyto6 commented Nov 4, 2022

gyto6 commented Nov 4, 2022

lastzero commented Nov 4, 2022

doctorttt commented Jul 1, 2023

lastzero commented Aug 13, 2022 •

edited

Loading

ngoldack commented Sep 19, 2022 •

edited

Loading

mlapaglia commented Sep 19, 2022 •

edited

Loading

Problem `-level:v 41`

jahanson commented Nov 3, 2022 •

edited

Loading

jahanson commented Nov 3, 2022 •

edited

Loading

gyto6 commented Nov 4, 2022 •

edited

Loading

gyto6 commented Nov 4, 2022 •

edited

Loading