Not working gpu transcode

[owerrride]

The bug

Processing on the 1050ti GPU does not turn on.
I added changes to docker-compose and turned on Nvidia in the settings accordingly, but the card is still constantly idle (
Driver installed, version 560.35.03

The OS that Immich Server is running on

Unraid 6.12.12

Version of Immich Server

1.114.0

Version of Immich Mobile App

1.114.0

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    devices:
      - /dev/dri:/dev/dri
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:e3b17ba9479deec4b7d1eeec1548a253acc5374d68d3b27937fcfe4df8d18c7e
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      #start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/user/Immich
# The location where your database files are stored
DB_DATA_LOCATION=/mnt/user/appdata/immich-db

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD===================

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. I make changes in docker-compose file
2. Change settings in immich web

Relevant log output

[Nest] 7  - 09/10/2024, 6:59:28 AM   ERROR [Microservices:MediaRepository] ffmpeg version 6.0.1-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Routing option strict to both codec and muxer layer
[AVHWDeviceContext @ 0x380e6030280] Cannot load libcuda.so.1
[AVHWDeviceContext @ 0x380e6030280] Could not dynamically load CUDA
Device creation failed: -1.
Failed to set value 'cuda=cuda:0' for option 'init_hw_device': Operation not permitted
Error parsing global options: Operation not permitted

[Nest] 7  - 09/10/2024, 6:59:28 AM   ERROR [Microservices:MediaService] Error: ffmpeg exited with code 1: Device creation failed: -1.
Failed to set value 'cuda=cuda:0' for option 'init_hw_device': Operation not permitted
Error parsing global options: Operation not permitted

[Nest] 7  - 09/10/2024, 6:59:28 AM   ERROR [Microservices:MediaService] Error occurred during transcoding. Retrying with NVENC acceleration disabled.

Additional information

No response

[Mraedis]

What is the group of of the cards in /dev/dri/? Try adding that group to the container you want to use HWA with.
Something like this:

    group_add:
      - video
      - render
    devices:
      - /dev/dri:/dev/dri

[owerrride]

Какова группа карт в /dev/dri/? Попробуйте добавить эту группу в контейнер, с которым вы хотите использовать HWA. Что-то вроде этого:

    group_add:
      - video
      - render
    devices:
      - /dev/dri:/dev/dri

immich server dont start with this
"Error response from daemon: Unable to find group render: no matching entries in group file"

[owerrride]

i have this structure, i tryed in compose file write

devices:
      - /dev/dri/card0:/dev/dri

and

devices:
      - /dev/dri/renderD128:/dev/dri

but i also have mistake

[owerrride]

now my docker-compose look that:

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    devices:
      - /dev/dri:/dev/dri
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:e3b17ba9479deec4b7d1eeec1548a253acc5374d68d3b27937fcfe4df8d18c7e
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      #start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

i am deleted from container machine-learning my video card

[Mraedis]

Какова группа карт в /dev/dri/? Попробуйте добавить эту группу в контейнер, с которым вы хотите использовать HWA. Что-то вроде этого:

    group_add:
      - video
      - render
    devices:
      - /dev/dri:/dev/dri

immich server dont start with this "Error response from daemon: Unable to find group render: no matching entries in group file"

Just use video then, it was an example of what it might look like on your system. :)

For passthrough, either mount the directory exactly, like you did before, with /dev/dri:/dev/dri or the renderer/card exactly like /dev/dri/renderD128:/dev/dri/renderD128: , don't try to mix the paths up.

[owerrride]

Для сквозного доступа либо смонтируйте каталог точно так же, как вы делали раньше, /dev/dri:/dev/driлибо рендерер/карту точно так же, как /dev/dri/renderD128:/dev/dri/renderD128:, не пытайтесь перепутать пути.

 immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

I make this but have mistakes in logs

[Nest] 7  - 09/10/2024, 9:49:54 AM     LOG [Microservices:MediaService] Started encoding video 7458727d-e6d2-4ac2-a88e-bc1f290e832a {"inputOptions":["-init_hw_device cuda=cuda:0","-filter_hw_device cuda"],"outputOptions":["-tune hq","-qmin 0","-rc-lookahead 20","-i_qfactor 0.75","-c:v h264_nvenc","-c:a aac","-movflags faststart","-fps_mode passthrough","-map 0:0","-strict unofficial","-map 0:1","-g 256","-v verbose","-vf format=nv12,hwupload_cuda","-preset p1","-cq:v 23"],"twoPass":false}
[Nest] 7  - 09/10/2024, 9:49:55 AM   ERROR [Microservices:MediaRepository] ffmpeg version 6.0.1-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Routing option strict to both codec and muxer layer
[AVHWDeviceContext @ 0x2cd0a030280] Cannot load libcuda.so.1
[AVHWDeviceContext @ 0x2cd0a030280] Could not dynamically load CUDA
Device creation failed: -1.
Failed to set value 'cuda=cuda:0' for option 'init_hw_device': Operation not permitted
Error parsing global options: Operation not permitted

[Nest] 7  - 09/10/2024, 9:49:55 AM   ERROR [Microservices:MediaService] Error: ffmpeg exited with code 1: Device creation failed: -1.
Failed to set value 'cuda=cuda:0' for option 'init_hw_device': Operation not permitted
Error parsing global options: Operation not permitted

[Nest] 7  - 09/10/2024, 9:49:55 AM   ERROR [Microservices:MediaService] Error occurred during transcoding. Retrying with NVENC acceleration disabled.
[Nest] 7  - 09/10/2024, 9:49:55 AM     LOG [Microservices:MediaService] Successfully generated WEBP video thumbnail for asset 7458727d-e6d2-4ac2-a88e-bc1f290e832a
[Nest] 7  - 09/10/2024, 9:50:15 AM     LOG [Microservices:MediaService] Successfully encoded 7458727d-e6d2-4ac2-a88e-bc1f290e832a

[Mraedis]

What user is docker running as? Try adding that user to the video group. For instance usermod -aG video docker to add the docker user to the video group. (Maybe you'll need sudo.)

[owerrride]

Под каким пользователем работает docker? Попробуйте добавить этого пользователя в группу video. Например, usermod -aG video dockerчтобы добавить пользователя docker в группу video. (Возможно, вам понадобится sudo.)

Didn't help, same error

Its write?

 immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    devices:
      - /dev/dri:dev/dri
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

[Mraedis]

You still need the group section in your compose, even after adding the group.

[owerrride]

You still need the group section in your compose, even after adding the group.

i am add group video:

immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    group_add:
      - video
    devices:
      - /dev/dri:dev/dri
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

but it not work, same mistake in logs.
user is root (what start docker container), he added to group video

[Mraedis]

What is the output of nvidia-smi on your host?

[owerrride]

I figured it out, everything worked with this:

Need add to docker-compose file in immich-server

group_add:
     - video
   deploy:
     resources:
       reservations:
         devices:
           - capabilities: [gpu]
   runtime: nvidia
   devices:
     - /dev/nvidia0:/dev/nvidia0
     - /dev/nvidiactl:/dev/nvidiactl
     - /dev/nvidia-uvm:/dev/nvidia-uvm
     - /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools

and this to .env file:

NVIDIA_VISIBLE_DEVICES=all
NVIDIA_DRIVER_CAPABILITIES=all

[Mraedis]

Thanks for letting us know. 👍

[yodaphone]

I figured it out, everything worked with this:

Need add to docker-compose file in immich-server

group_add:
     - video
   deploy:
     resources:
       reservations:
         devices:
           - capabilities: [gpu]
   runtime: nvidia
   devices:
     - /dev/nvidia0:/dev/nvidia0
     - /dev/nvidiactl:/dev/nvidiactl
     - /dev/nvidia-uvm:/dev/nvidia-uvm
     - /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools

and this to .env file:

NVIDIA_VISIBLE_DEVICES=all
NVIDIA_DRIVER_CAPABILITIES=all

Hi can you please share your full docker compose file & env file with necessary redacted info? I'm trying to set this up in docker in windows with WSL2 and i'm not able to get it right

[Mraedis]

This is not docker on windows at all so it is not very relevant for you. Please either make your own discussion with YOUR compose or do the same one discord. Check out https://immich.app/docs/features/ml-hardware-acceleration as well.

[Werner0101]

Hey,
i have a similar issue.

Before i go deeper i am new to server hosting -> i dont have any f clue what i am doing

instead of unraid i have a proxmox server running an LXC (unprivileged) with the docker version of the current (2.2.3) version of immich
The GPU passthrough to the LXC is ok both nvtop and nvidia-smi are working and showing my 1060 GPU

Still i get the same errors as above.

I tried adding the stuff from the answer to my docker yaml but this doesnt work.
It still transcodes on the CPU.

My yaml currently looks like this:

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
      group_add:
        - video
      deploy:
        resources:
          reservations:
            devices:
              - capabilities: [gpu]
      runtime: nvidia
        devices:
          - /dev/nvidia0:/dev/nvidia0
          - /dev/nvidiactl:/dev/nvidiactl
          - /dev/nvidia-uvm:/dev/nvidia-uvm
          - /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools
   # extends:
   #   file: hwaccel.transcoding.yml
   #   service: nvenc # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/data
      - /etc/localtime:/etc/localtime:ro
    env_file:

i tried with both the extends enabled or not but it doesnt make a difference.
At this point i am out of ideas and after the latest docker update wiped my whole system i am quitre done with the HW transcodeing