Server does not start after upgrade

[gbcatrinoiu]

The bug

I upgraded from v1.98.x to v1.106.1 following the breaking changes from the latest release. I did not want to opt-in for the changes from v1.102.0 so I ignored those.

I did a rebuild of my docker-compose file and now the server won't start.

The OS that Immich Server is running on

DSM 7.2

Version of Immich Server

v1.106.1

Version of Immich Mobile App

v1.106.3

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - ${EXTERNAL_PATH}:/usr/src/app/external
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=[redacted]

# The external location
EXTERNAL_PATH=[redacted]

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=[redacted]

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=[redacted]
DB_USERNAME=[redacted]
DB_DATABASE_NAME=[redacted]

REDIS_HOSTNAME=[redacted]

Reproduction steps

1. Build the docker-compose-yaml file
2. Try to access the server

Relevant log output

There are no logs since the container won't start.

Additional information

No response

[bo0tzz]

Did you pull the new images as well?

[gbcatrinoiu]

yes, I did

[gbcatrinoiu]

I also did a docker-compose pull from the cli but nothing changes

[MexHigh]

No logs? If you run docker compose logs, is there a exit code ("container XY exited with code XY")? Looks like a docker issue to me

[gbcatrinoiu]

I only get logs related to the other 3 containers, nothing related to the server container. I believe it simply does not start.

[MexHigh]

Try: docker compose ps -a and see if the container restarts periodically (uptime is smaller compared to the other containers).

Also try: docker compose down --remove-orphans. Maybe there is another old server containing creating conflicts?

Did you try to restart the docker daemon or the whole system?

[hersa37]

If you don't run in detached mode, do you see the postgres container restarting over and over again?

[jhjaggars]

I'm running on kubernetes and installed via the helm chart and the microservices container is crashlooping with the following error:
/bin/sh: 0: cannot open ./start-microservices.sh: No such file

The main server container is failing with the following error:
/bin/sh: 0: cannot open ./start-server.sh: No such file

Maybe the image build is bad?

Image:         ghcr.io/immich-app/immich-server:v1.106.4                                                                                                                                                                                                        
Image ID:      ghcr.io/immich-app/immich-server@sha256:d39cb7ecbcc9924f2c51a3e0deb8a469075996c6ba9e1384eb2ddb550984848e

❯ podman run -it ghcr.io/immich-app/immich-server@sha256:d39cb7ecbcc9924f2c51a3e0deb8a469075996c6ba9e1384eb2ddb550984848e ls
/usr/bin/ls: /usr/bin/ls: cannot execute binary file

I just realized that there were some changes in 1.106 that moved the way microservices starts. Running helm repo update and then applying the chart again fixes the issue for me.