immich-machine-learning openvino not working (transcoding works)

[Str1atum]

The bug

Hi all,

I configured ubuntu in docker on Ubuntu 24.04 on a proxmox VM with PCIe passthrough for my Intel 13th gen Iris Xe
So far intel_gpu_top and hardware transcoding is working fine in immich (can see the utilization in intel_gpu_top and speed is fine) but I cannot get the ml part to use the GPU with openvino
The error in the logs is just:
WARNING No GPU device found in OpenVINO. Falling back to CPU

The OS that Immich Server is running on

Ubuntu 24.04

Version of Immich Server

v1.105.1

Version of Immich Mobile App

v1.105.1

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: openvino # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: unless-stopped

Your .env content

standard .env. just with folders / passwords changed

Reproduction steps

see above

Relevant log output

WARNING No GPU device found in OpenVINO. Falling back to CPU

Additional information

No response

[mertalev]

I'm guessing you're either missing a driver, or 24.04 is too new for the OpenVINO version in the container. Also, what model is the CPU? I last tested it on a 13700H.

[Str1atum]

It's i9-13900H

I tried with bookworm Debian 6.1 kernel but had no success at all.

[mertalev]

Hmm, you can try looking at some of the drivers mentioned in this page. This page also suggests using --group-add. For Compose files, this would mean first running stat -c \"%g\" /dev/dri/render*, then adding the output as:

...
group_add:
  - <output>

Also, for what it's worth, I tested on Fedora.

[Str1atum]

No success - driver installation went smooth but no changes.
And I added this to my hwaccel.ml.ml

openvino:
device_cgroup_rules:
- "c 189:* rmw"
devices:
- /dev/dri:/dev/dri
group_add:
- "993"
volumes:
- /dev/bus/usb:/dev/bus/usb

[mertalev]

What's the output of ls -l /dev/dri? Also, can you set LOG_LEVEL=debug, try again and find the line Available OpenVINO devices: in the ML logs?

[Str1atum]

ls -l /dev/dri:

total 0
drwxr-xr-x 2 root root 100 May 16 14:49 by-path
crw-rw---- 1 root video 226, 0 May 16 14:49 card0
crw-rw---- 1 root video 226, 1 May 16 14:49 card1
crw-rw---- 1 root render 226, 128 May 16 14:49 renderD128

[mertalev]

Can you also add the output of stat -c \"%g\" /dev/dri/card* to group_add? I think the OpenVINO image is non-root, so it may not be able to access the devices in /dev/dri.

[Str1atum]

but still the same error

[mertalev]

Try adding this:

    environment:
      - NEOReadDebugKeys=1
      - OverrideGpuAddressSpace=48

[Str1atum]

that seems to have worked

[mertalev]

Sweet! That means it's a bug in intel-compute-runtime.

[Str1atum]

So anything we can do about that or do I just need to keep that compose extension forever?

[mertalev]

We would need the latest patched release for intel-compute-runtime in the image, which will take some time to be available in package managers. You can just keep those envs around until then.

[mertalev]

I decided to add these to the hwaccel.ml.yml file in the meantime: #9541.

[vuongtt92]

I added this to the openvino section in hwaccel.ml.yml but it still doesn't work. Mine is intel N5015 installing Xpenology 7.2. Could you please help? The log said "no gpu device found in openvino"
```
environment:
- NEOReadDebugKeys=1
- OverrideGpuAddressSpace=48

[mertalev]

That fix only applies to kernels 6.7.5 or newer. I don't imagine a Synology server would have such a new kernel. It could be that the kernel is too old instead and/or that you're missing a driver.

[vuongtt92]

so there's nothing I can do about it? I see that hardware accelerator works well with Plex while transcoding but I've tried many fixes but can't get it work with immich ML

[mertalev]

I can't say for certain, but I wouldn't be surprised if it's impossible without a newer host environment.

But on the other hand, your processor would likely run into other issues with OpenVINO, so you may have had to use CPU instead anyway. I can only recommend OpenVINO for Iris Xe and Arc graphics - anything else is too unreliable from what I've seen.