Machine Learning container crashing after upgrade from 2.5.6 to 2.6.1

[drakz-AU]

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.

• Yes

The bug

After updating to the latest version of Immich the Machine Learning container is crashing and restarting constantly.

System console is full of lines like this:

2026 Mar 21 18:24:46 nas Process 1587482 (python) of user 0 dumped core.

Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libquadmath-96973f99-934c22de.so.0.0.0 without build-id.
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libquadmath-96973f99-934c22de.so.0.0.0
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libgfortran-040039e1-0352e75f.so.5.0.0 without build-id.
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libgfortran-040039e1-0352e75f.so.5.0.0
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libscipy_openblas64_-096271d3.so without build-id.
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libscipy_openblas64_-096271d3.so
Module /opt/venv/lib/python3.11/site-packages/numpy/_core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so without build-id.
Module /opt/venv/lib/python3.11/site-packages/numpy/_core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so
Stack trace of thread 7:
#0  0x00007fce1b061cec n/a (/opt/venv/lib/python3.11/site-packages/numpy/_core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so + 0x2dcec)
#1  0x00007fce1e1a19ce n/a (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 + 0x49ce)
ELF object binary architecture: AMD x86-64

Going back to version 2.5.6 it works correctly again.

The OS that Immich Server is running on

TrueNAS Scale 25.10.2.1

Version of Immich Server

v2.6.1

Version of Immich Mobile App

2.5.6 build.3037

Platform with the issue

• Server
• Web
• Mobile

Device make and model

No response

Your docker-compose.yml content

#
# WARNING: To install Immich, follow our guide: https://docs.immich.app/install/docker-compose
#
# Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/data
      - /mnt/uberspace/Immich/library:/data/library
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - '2283:2283'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://docs.immich.app/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/valkey/valkey:9@sha256:3eeb09785cd61ec8e3be35f8804c8892080f3ca21934d628abc24ee4ed1698f6
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
      # Uncomment the DB_STORAGE_TYPE: 'HDD' var if your database isn't stored on SSDs
      # DB_STORAGE_TYPE: 'HDD'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    shm_size: 128mb
    restart: always
    healthcheck:
      disable: false

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://docs.immich.app/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/uberspace/Immich/data

# The location where your database files are stored. Network shares are not supported for the database
DB_DATA_LOCATION=/mnt/uberspace/Immich/pgData

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v2.1.0"
IMMICH_VERSION=v2

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=###REDACTED###

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=immich
DB_DATABASE_NAME=immich

Reproduction steps

1. Update from v2.5.6 to v2.6.1 on TrueNAS
2. Machine Learning container starts crashing

Relevant log output

2026 Mar 21 18:24:46 nas Process 1587482 (python) of user 0 dumped core.

Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libquadmath-96973f99-934c22de.so.0.0.0 without build-id.
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libquadmath-96973f99-934c22de.so.0.0.0
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libgfortran-040039e1-0352e75f.so.5.0.0 without build-id.
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libgfortran-040039e1-0352e75f.so.5.0.0
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libscipy_openblas64_-096271d3.so without build-id.
Module /opt/venv/lib/python3.11/site-packages/numpy.libs/libscipy_openblas64_-096271d3.so
Module /opt/venv/lib/python3.11/site-packages/numpy/_core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so without build-id.
Module /opt/venv/lib/python3.11/site-packages/numpy/_core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so
Stack trace of thread 7:
#0  0x00007fce1b061cec n/a (/opt/venv/lib/python3.11/site-packages/numpy/_core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so + 0x2dcec)
#1  0x00007fce1e1a19ce n/a (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 + 0x49ce)
ELF object binary architecture: AMD x86-64

Additional information

Potentially related: this is an old system that does not support AVX. I mention this as I have recently had problems with this on a couple of other containers.
CPU is an AMD Phenom(tm) II X6 1055T
CPU flags fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt cpb hw_pstate vmmcall npt lbrv svm_lock nrip_save pausefilter

[mmomjian]

The Phenom II (17 years old) has a K10 micro architecture which does not support x86-64-v2. Numpy removed support for x86-64-v1 in v2.4.0.

https://numpy.org/devdocs/release/2.4.0-notes.html

[Fox350]

I had the same problem using docker under linux vm.
Got the idea to revisit my Vmware EVC mode for the cluster because I did retire an older esxihost from the cluster.
I Was able to change my evc mode from CPU Merom to Nehalem, shutdown the vm, power it on so OS see the new exposed features.
This is for me the best solution and quick fix but I thing those kind of compilation should be done considering most homelab user are using old hardware to run them.
In all cases, thanks a lot to all the team to their great job !

[mmomjian]

We are discussing with the team how to handle this. Most people are able to update their VM cpu set as mentioned, which is honestly a good thing. Users on bare metal may have to return to immich 2.5

[BWagener]

I have a similar issue since upgrading to v2.6.x. with the specific error message being:

RuntimeError: NumPy was built with baseline optimizations: (X86_V2) but your machine doesn't support: (X86_V2).

For Proxmox users this article shows how to change the VM CPU type: https://www.techaddressed.com/tutorials/proxmox-improve-vm-cpu-perf/#on-existing-v-ms

Changing it to "host" fixed it for me.

[tuyenhn]

Changing it to "host" fixed it for me.

FYI, arbitrarily changing it to x86-64-v2 (and v3, v4) also works. This fixed it on my Proxmox VM

[drakz-AU]

Can the machine learning container be locked to version 2.5.6, while updating the server to version 2.6.1?
Or do the server and machine learning containers need to be the same version?

Would it be possible to support a 'legacy' build for users with older systems?

I guess ultimately it's time for an upgrade, I've been running into more of these problems lately, but it's hard to justify the cost of new hardware when this still works for my needs (compatibility issues aside). ¯\(ツ)/¯

[mmomjian]

For now, ML 2.5 should still work with immich 2.6, but that’s not always the case.

It’s unlikely we would create a separate build for such a niche use case. Sorry. You could do this yourself if you want.

[drakz-AU]

For anyone else who may be having this issue, one workaround could be running the Machine Learning container on a different system: https://docs.immich.app/guides/remote-machine-learning/

[hKaspy]

I have a similar issue since upgrading to v2.6.x. with the specific error message being:

RuntimeError: NumPy was built with baseline optimizations: (X86_V2) but your machine doesn't support: (X86_V2).

For Proxmox users this article shows how to change the VM CPU type: https://www.techaddressed.com/tutorials/proxmox-improve-vm-cpu-perf/#on-existing-v-ms

Changing it to "host" fixed it for me.

Thanks for that! Can confirm the same for a TrueNAS Scale VM.

I have a Skylake CPU and for some reason when creating the VM Immich is on I chose "custom" for the CPU Mode. Editing the VM config in UI and changing to "Host Model" was enough 😄

[timonrieger]

closing this as stale. reopen if still relevant