Questions on [Workaround] / [Future Support] v3 Immich Machine Learning with Maxwell (GTX 9xx) and Pascal (GTX 10xx) series (Compute Capability 5.2)

[Ramen-LadyHKG]

TLDR

My main concerns are:

1. Will Maxwell/Pascal be ignored in v3 [Immich ML]?
2. If so, does recompiling the ONNX Runtime ourselves fix the problem?
3. Alternatively, can the old ML container (immich-machine-learning:v2.4.1-cuda) remain compatible with the immich-server v3?
4. What will happen to the already computed assets? Will we have to rerun all machine learning tasks (CLIP, face recognition, etc.)?

v3 Upgrade news

Immich v3 introduces a lot of exciting upgrades (better sharing and shared features, Workflows, Smart Albums, etc.), as seen in the Immich Blog - v3 release candidate, Immich Blog - April recap, and the Immich Roadmap.

Concerns regarding old GPU support for Machine Learning

However, I haven't thoroughly looked into future support for Machine Learning on older GPU architectures.

As far as I know, the Immich Blog - v3 Migration guide only mentions CPU microarchitecture requirements:

v3 migration Machine learning - Hardware requirements

A bump in numpy now requires x86 CPUs to be in the x86-64-v2 microarchitecture level or higher. This does not mean AVX is required; virtually every mainstream x86 CPU since 2010 meets the x86-64-v2 baseline.

My current build

I am currently running Immich on an older PC.

• Hardware: i7-4790 | 16GB RAM | GTX 980 Ti 6GB VRAM
• Versions: Driver: 580.159.04 | CUDA: 13.0
• OS: CachyOS (7.0.12-1-cachyos)

Docker Images:

• Immich Server: 2.7.5
• Immich Machine Learning: v2.4.1-cuda

Models (Immich Machine Learning):

• CLIP: ViT-SO400M-16-SigLIP2-384__webli
• Face recognition: antelopev2
• OCR: PP-OCRv5_server

My current docker-compose

Click to expand code

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: nvenc
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      
# 1. RELOCATE POOLED THUMBNAILS MAPPING ...
      - /$HOME/mounted/immich-data/library/immich_encoded-video-MERGEFS:/usr/src/app/upload/encoded-video
      - /$HOME/mounted/immich-data/library/immich_thumbs_MERGEFS:/usr/src/app/upload/thumbs
      - /$HOME/mounted/immich-data/library/immich_backups_MERGEFS:/usr/src/app/upload/backups
      - /$HOME/mounted/immich-data/library/immich_backups_MERGEFS:/usr/src/app/upload/backups
      
# 2. EXTERNAL MEDIA PHOTO PATHS (READ ONLY) ...
      - "/$HOME/mounted/[Media]/Photos/:/mnt/media/Photos/:ro"

    env_file:
      - .env
    ports:
      - '8888:8888'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

# NEW LEGACY CONTAINER FOR THE 980ti
  immich-ml-legacy:
    container_name: immich_ml_legacy
    image: ghcr.io/immich-app/immich-machine-learning:v2.4.1-cuda
    network_mode: host  # Force the container to use my host's network and DNS directly
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    environment:
      - IMMICH_MACHINE_LEARNING_DEVICE=cuda
      - NVIDIA_VISIBLE_DEVICES=all  # Hard-set to expose both; safer than ${...}
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility,video
    ports:
      - "8883:8883"  # Expose the internal container port to the host machine
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: always
      
  database:
    container_name: immich_postgres
    image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
      POSTGRESQL_AUTOVACUUM_VACUUM_SCALE_FACTOR: 0.02
      POSTGRESQL_AUTOVACUUM_ANALYZE_SCALE_FACTOR: 0.01
      POSTGRESQL_AUTOVACUUM_NAPTIME: 30s
      POSTGRESQL_AUTOVACUUM_VACUUM_COST_LIMIT: 1000
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    shm_size: 128mb
    restart: always
    healthcheck:
      disable: false

volumes:
  model-cache:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: "/$HOME/mounted/immich-data/immiach_docker_image/"

Maxwell/Pascal support dropped from ONNX Runtime for sm_52 or fails after Immich v2.5.2

(Workaround) Downgrading "Immich Machine Learning" to v2.4.1

Although Compute Capability 5.2 (GTX 9xx series) was listed as supported, Immich Machine Learning refuses to work on Maxwell/Pascal cards.

This issue was discussed here.

I understand that ONNX Runtime can no longer maintain support for these older architectures within a single image, meaning a manual recompilation is required.

Our current workaround has been to stick with an older ML image (image: ghcr.io/immich-app/immich-machine-learning:v2.4.1-cuda).

Cheers all!