Facial Recognition re-running every day

[alexperjescu]

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.

• Yes

The bug

Immich completes the Facial Recognition process for all photos. Then, if I click the Missing button, or automatically every early morning, Immich will start the Facial Recognition process again for 20,000 to 50,000 photos. This would happen every day if left alone.

I tried resetting the Face Detection. Immich scanned all photos again, and then ran the Facial Recognition process, but the issue still occurs.

The OS that Immich Server is running on

Debian 12

Version of Immich Server

1.132.1

Version of Immich Mobile App

1.132.0

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich
services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      #  - immichData:/usr/src/app/upload
      - /mnt:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:2283
    depends_on:
      - redis
      - database
    restart: always
  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      #  - model-cache:/cache
      - immichRedis:/cache
    env_file:
      - .env
    restart: always
  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:328fe6a5822256d065debb36617a8169dbfbd77b797c525288e465f56c1d392b
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: --data-checksums
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' ||
        exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}'
        --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT
        COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo
        "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command:
      - postgres
      - -c
      - shared_preload_libraries=vectors.so
      - -c
      - search_path="$$user", public, vectors
      - -c
      - logging_collector=on
      - -c
      - max_wal_size=2GB
      - -c
      - shared_buffers=512MB
      - -c
      - wal_compression=on
    restart: always
volumes:
  immichData:
    external: true
  immichRedis:
    external: true
  immichDatabase:
    external: true
  model-cache: null
networks: {}

Your .env content

DB_DATABASE_NAME=***
DB_PASSWORD=***
DB_USERNAME=***
DB_DATA_LOCATION=/var/lib/postgresql/data

Reproduction steps

1. I believe that this has started happening after restoring the database from a backup

Relevant log output

Additional information

No response

[bo0tzz]

How many assets do you have in your instance? It's normal for this job to run every night to make sure the recognition algorithm is all tidied up.

[alexperjescu]

How many assets do you have in your instance? It's normal for this job to run every night to make sure the recognition algorithm is all tidied up.

I have 262,000 photos and 7,200 videos.

After resetting the facial detection and rescanning all the assets for facial detection and facial recognition, when I pressed the Missing button it said it had to scan 53,000 assets. Is that normal behaviour? That process alone could take 24-36 hours.

[bo0tzz]

I believe this is normal and it should be quick. cc @mertalev

[alexperjescu]

I believe this is normal and it should be quick. cc @mertalev

thanks. For context, it has completed about 3000 assets in about one hour.

Immich is running on a Proxmox VM with 6 CPUs and 64GB RAM.

[mertalev]

So it's averaging less than 1 face per second? That's definitely much slower than it should be. It's possible you're missing the index that makes this quick.

Try opening a DB session with docker exec -it immich_postgres psql -U postgres immich and running this command:

CREATE INDEX face_index ON face_search
USING vectors (embedding vector_cos_ops)
WITH (options = $$
[indexing.hnsw]
m = 16
ef_construction = 300
$$);

[mertalev]

Be warned that it can take some minutes to finish, and the index will do more processing in the background before it can be used. Wait for Postgres to go idle before retesting.

[alexperjescu]

So it's averaging less than 1 face per second? That's definitely much slower than it should be. It's possible you're missing the index that makes this quick.

Try opening a DB session with docker exec -it immich_postgres psql -U postgres immich and running this command:

CREATE INDEX face_index ON face_search
USING vectors (embedding vector_cos_ops)
WITH (options = $$
[indexing.hnsw]
m = 16
ef_construction = 300
$$);

Thanks I think that's done it, when I hit the Missing button it came up with 53,000 assets but the scan was much quicker, it seems to complete the whole lot in about 30 minutes. Also, the CPU usage seems to be about half of what it was previously.

[agusdmb]

i am having this issue after updating to v1.133.0 and v1.133.1, i have tryied adding that index but im guessing with the change of the DB in the latest version is not the same index now (i tried and it doesn't work)

immich=# CREATE INDEX face_index ON face_search
USING vectors (embedding vector_cos_ops)
WITH (options = $$
[indexing.hnsw]
m = 16
ef_construction = 300
$$);
ERROR:  access method "vectors" does not exist
immich=# Shared connection to docker closed.

@mertalev do you know which index should i be creating?

[mertalev]

1.133 has more comprehensive checks here and will create missing indices on startup, so you don't need to do anything. Can you clarify what behavior you're seeing?

[agusdmb]

so the "facial recognition" job was taking for ever, it actually ran every 24 hours and took more than 6 hours each time for 5000 pics..

i got to fix it, what i did was to open a shell on postgres and ran: ANALYZE face_search;
that fixed it, now it does all 5k photos in seconds

[vb0]

i got to fix it, what i did was to open a shell on postgres and ran: ANALYZE face_search; that fixed it, now it does all 5k photos in seconds

This fixed it, well done! After one of the recent updates I got to wait some days (or maybe over a week?) for "FACIAL RECOGNITION" to finish on like 30-40k items. I was considering upgrading the machine to a real monster, but it was just postgres keeping one core 100%, and while I can multiply easily the total performance or RAM many times single core it's very hard to improve a lot.

Anyway it was done, today I added just a few pics and started again from scratch, at the same glacial speed! After ANALYZE it picked up speed probably with a factor of 100-1000 and finished in no time!