I have a error on face recognizing

[reheinz]

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.

• Yes

The bug

I have a error on face recognizing

The OS that Immich Server is running on

Synology

Version of Immich Server

v.1.132.1 to 1.132.3

Version of Immich Mobile App

v1.132.3

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - ${EXTERNAL_PATH}:/usr/src/app/TEST
      - /volume3/photo/01_Hj_So:/usr/src/app/01_Hj_So
    env_file:
      - stack.env
    ports:
      - 2283:2283
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - ${MODEL_CACHE}:/cache
    env_file:
      - stack.env
    restart: always
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    healthcheck:
      disable: false
    
  redis:
    container_name: immich_redis
    image: docker.io/valkey/valkey:8-bookworm@sha256:42cba146593a5ea9a622002c1b7cba5da7be248650cbb64ecb9c6c33d29794b1
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
    labels:
      - "com.centurylinklabs.watchtower.enable=false"

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:739cdd626151ff1f796dc95a6591b55a714f341c737e27f045019ceabf8e8c52
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: >-
        pg_isready --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" || exit 1; 
        Chksum="$$(psql --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" --tuples-only --no-align
        --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')";
        echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
        [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
#     start_interval: 30s            Need Docker Engine v25
      start_period: 5m
      
    command: >-
      postgres
      -c shared_preload_libraries=vectors.so
      -c 'search_path="$$user", public, vectors'
      -c logging_collector=on
      -c max_wal_size=2GB
      -c shared_buffers=512MB
      -c wal_compression=on
      
    restart: always
    labels:
      - "com.centurylinklabs.watchtower.enable=false"

Your .env content

UPLOAD_LOCATION=/volume3/docker/04_immich/library
EXTERNAL_PATH=/volume3/photo/TEST
DB_DATA_LOCATION=/volume3/docker/04_immich/pgdata
MODEL_CACHE=/volume3/docker/04_immich/model-cache
IMMICH_VERSION=release
DB_PASSWORD=............
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
REDIS_HOSTNAME=immich_redis
TZ=Asia/Seoul
REDIS_DATA=/volume3/docker/04_immich/redis
PGTZ=Asia/Seoul

Reproduction steps

1. After update v1.132.1
2. I can find Something changed redis and database version.
3. I changed the database ver. and redis ver.
4. 4
5. After My server can't recognize face about additional pictures.
   ...

Relevant log output

[Nest] 7  - 04/30/2025, 5:37:30 AM   ERROR [Microservices:{"id":"3bfe0e49-6fea-465f-a8e5-be0c8d26f995"}] Unable to run job handler (faceDetection/face-detection): PostgresError: pgvecto.rs: Bad literal.
INFORMATION: hint = Bad character with ascii 0x30.
PostgresError: pgvecto.rs: Bad literal.
INFORMATION: hint = Bad character with ascii 0x30.
    at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:790:26)
    at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:476:6)
    at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
    at Socket.emit (node:events:518:28)
    at addChunk (node:internal/streams/readable:561:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
    at Readable.push (node:internal/streams/readable:392:5)
    at TCP.onStreamRead (node:internal/stream_base_commons:189:23)

Additional information

No response

[bo0tzz]

Do you use a remote machine-learning container? This error usually indicates that the ML image is out of date.

[reheinz]

Do you use a remote machine-learning container? This error usually indicates that the ML image is out of date.

You can see the compose file and env. I use the latest released ML image.

[bo0tzz]

That doesn't answer my question. Can you share a screenshot of your ML configuration from https://my.immich.app/admin/system-settings?isOpen=machine-learning ?

[reheinz]

That doesn't answer my question. Can you share a screenshot of your ML configuration from https://my.immich.app/admin/system-settings?isOpen=machine-learning ?

Could you see this setting? Sorry it's in Korean language.

[reheinz]

It seems that the minus sign and the place before the decimal point are not recognized as numbers, but as letters. I think it's a parsing error, but I don't know how to solve it.

1. Error log infomation :

INFORMATION: hint = Bad character with ascii 0x2d.
INFORMATION: hint = Bad character with ascii 0x30.
INFORMATION: hint = Bad character with ascii 0x31.

===============================================================

2. Example :

Query failed : {
durationMs: 10.86070699999982,
error: PostgresError: pgvecto.rs: Bad literal.
INFORMATION: hint = Bad character with ascii 0x30.
at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:790:26)
at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:476:6)
at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
at Socket.emit (node:events:518:28)
at addChunk (node:internal/streams/readable:561:12)
at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
at Readable.push (node:internal/streams/readable:392:5)
at TCP.onStreamRead (node:internal/stream_base_commons:189:23) {
severity_local: 'ERROR',
severity: 'ERROR',
code: 'XX000',
where: "unnamed portal parameter $18 = '...'",
file: 'error.rs',
line: '7'
},
sql: 'with "added" as (insert into "asset_faces" ("id", "assetId", "imageHeight", "imageWidth", "boundingBoxX1", "boundingBoxY1", "boundingBoxX2", "boundingBoxY2") values ($1, $2, $3, $4, $5, $6, $7, $8), ($9, $10, $11, $12, $13, $14, $15, $16)), "added_embeddings" as (insert into "face_search" ("faceId", "embedding") values ($17, $18), ($19, $20)) select from (select 1) as "dummy"',
params: [
'efeb0f35-80fa-4d4c-97d2-e0526125863e',
'6ef14dd7-d9f4-4e50-995f-fa007369c4ee',
�
,
�
,
810,
534,
�
,
900,
'5e8469a8-c5e8-4c83-bfb5-482408f680f3',
'6ef14dd7-d9f4-4e50-995f-fa007369c4ee',
�
,
�
,
�
,
481,
�
,
898,
'efeb0f35-80fa-4d4c-97d2-e0526125863e',
[
0.44447154, 0.4329499, -0.36303115, 0.09731434, 1.7801527,
0.7971078, 0.6464683, -0.991115, -1.1320091, 0.5329297,
-0.92926323, 1.0227695, -1.4600092, -2.1421747, 0.21701822,
2.058753, -1.1218532, 0.2612334, 0.41886854, -0.23774889,
-0.21500392, -0.01806509, -0.96538347, 1.1742127, -0.69650674,
-0.39686424, 0.9089627, -0.08469717, -0.18110496, -0.15159103,
-1.0021918, 0.23393303, -1.3799623, 1.0402119, 0.2289387,
1.1265018, -2.3204255, -0.79405516, 0.1691747, -2.117726,
0.7265288, 0.41961032, 0.20330006, 2.8758879, 0.34692678,
-0.79028505, 0.35620588, -1.2319635, 0.08902533, -0.24308783,
0.04916457, 1.5219285, 0.9976839, 1.574275, -0.51216066,
-0.5263979, -0.53500336, -1.681177, 0.8424934, -0.44701993,
0.49324226, -0.697721, 1.9756017, -0.5375532, -0.92460704,
1.2076886, 0.5829428, 0.37382, -0.015938073, -0.4969591,
-0.38491738, 1.431103, 0.33339706, -0.21465042, 1.6947784,
-0.6576294, 0.24504685, 0.0379193, -2.0425272, -0.28783616,
2.1378782, -2.0378776, -0.4486763, -0.5651556, 0.10389524,
0.48316902, -0.6617323, -0.90210384, 0.82726395, 1.4588242,
-0.41240332, 1.172142, 1.1531212, 1.0657201, -1.6241913,
0.5788297, 1.9497807, -1.2422631, 0.6292525, 0.50195956,
... 412 more items
],
'5e8469a8-c5e8-4c83-bfb5-482408f680f3',
[
-1.7215887, -0.62698287, 0.56639206, -0.1720142, -0.52667135,
-0.36877903, -0.14555748, -0.2164926, -0.14710665, 0.7187708,
-0.7250575, -0.21115932, -0.7805265, -1.0486385, -1.2347242,
-0.32018355, 0.993052, -0.577705, 0.73327786, -0.95860124,
-0.53962016, 1.7122722, 0.6526566, -0.7246237, -1.1155796,
-0.24901369, 1.8681824, 1.6934065, 0.95549184, -0.8475485,
1.7540313, -0.018414348, -2.0020616, 1.39733, -0.77626944,
0.5120844, -0.389606, 0.41024595, -0.51828945, 0.87703407,
0.44002312, -1.7428846, 0.35393688, 0.020943832, -1.115433,
-1.2687783, 0.8970287, -1.6924405, -0.5362458, 0.7242686,
0.9602648, 0.5985564, 0.48101643, 1.1923465, 0.67334193,
0.8778553, -1.4802591, -0.58529836, 0.8765532, -1.1515545,
0.5560353, 1.0093518, 0.06844661, -0.9936713, 1.8357918,
0.42010188, -1.4176073, 0.60510385, 0.5941961, -0.034416735,
-0.26305383, -0.86476797, 0.5365532, 1.2917768, 1.2906574,
-0.36676046, -0.18175465, -1.4043555, 1.700136, 0.9500369,
-0.13239285, -0.36222875, -0.65158045, -0.5470068, -0.2013168,
0.4007647, -0.22844896, 0.43126327, -0.5479686, -0.76148844,
-0.9348564, 0.75811905, -1.1055626, -0.114091456, 0.7273732,
0.16490617, -0.49810615, 0.23664802, -0.91529834, -0.9439533,
... 412 more items
]
]
}
[Nest] 7 - 05/09/2025, 11:21:03 AM ERROR [Microservices:{"id":"6ef14dd7-d9f4-4e50-995f-fa007369c4ee"}] Unable to run job handler (faceDetection/face-detection): PostgresError: pgvecto.rs: Bad literal.
INFORMATION: hint = Bad character with ascii 0x30.
PostgresError: pgvecto.rs: Bad literal.
INFORMATION: hint = Bad character with ascii 0x30.
at ErrorResponse (/usr/src/app/node_modules/postgres/cjs/src/connection.js:790:26)
at handle (/usr/src/app/node_modules/postgres/cjs/src/connection.js:476:6)
at Socket.data (/usr/src/app/node_modules/postgres/cjs/src/connection.js:315:9)
at Socket.emit (node:events:518:28)
at addChunk (node:internal/streams/readable:561:12)
at readableAddChunkPushByteMode (node:internal/streams/readable:512:3)
at Readable.push (node:internal/streams/readable:392:5)
at TCP.onStreamRead (node:internal/stream_base_commons:189:23)
orderBy(..., expr) is deprecated. Use orderBy(..., 'asc') or orderBy(..., (ob) => ...) instead.

[bo0tzz]

Can you post the output of docker inspect immich_machine_learning?

[reheinz]

Like below:

{
    "AppArmorProfile": "docker-default",
    "Args": [
        "--",
        "./start.sh"
    ],
    "Config": {
        "AttachStderr": true,
        "AttachStdin": false,
        "AttachStdout": true,
        "Cmd": [
            "./start.sh"
        ],
        "Domainname": "",
        "Entrypoint": [
            "tini",
            "--"
        ],
        "Env": [
            "DB_USERNAME=postgres",
            "PG_TZ=Asia/Seoul",
            "UPLOAD_LOCATION=/volume3/docker/04_immich/library",
            "EXTERNAL_PATH=/volume3/photo/TEST",
            "DB_DATA_LOCATION=/volume3/docker/04_immich/pgdata",
            "DB_PASSWORD=xxxxxxxxxxx",
            "TZ=xxxxxxxxl",
            "REDIS_DATA=/volume3/docker/04_immich/redis",
            "DB_HOSTNAME=immich_postgres",
            "REDIS_HOSTNAME=immich_redis",
            "MODEL_CACHE=/volume3/docker/04_immich/model-cache",
            "DB_DATABASE_NAME=immich",
            "PATH=/opt/venv/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "LANG=C.UTF-8",
            "GPG_KEY=A035C8C19219BA821ECEA86B64E628F8D684696D",
            "PYTHON_VERSION=3.11.10",
            "TRANSFORMERS_CACHE=/cache",
            "PYTHONDONTWRITEBYTECODE=1",
            "PYTHONUNBUFFERED=1",
            "PYTHONPATH=/usr/src",
            "DEVICE=cpu"
        ],
        "Healthcheck": {
            "Test": [
                "CMD-SHELL",
                "python3 healthcheck.py"
            ]
        },
        "Hostname": "a63fa5dabefa",
        "Image": "ghcr.io/immich-app/immich-machine-learning:release",
        "Labels": {
            "com.docker.compose.config-hash": "c8fd39bbc54e9353fbe255980e44e2384d45e6cb25c70c82effaf083edf786c7",
            "com.docker.compose.container-number": "1",
            "com.docker.compose.depends_on": "",
            "com.docker.compose.image": "sha256:5aeea9d29ed8edf8ce9abb7cf5c4f51976c6e4842d78553607514f54941aba55",
            "com.docker.compose.oneoff": "False",
            "com.docker.compose.project": "immich",
            "com.docker.compose.project.config_files": "",
            "com.docker.compose.project.working_dir": "/data/compose/17",
            "com.docker.compose.service": "immich-machine-learning",
            "com.docker.compose.version": "",
            "org.opencontainers.image.created": "2024-10-15T11:05:26.704Z",
            "org.opencontainers.image.description": "High performance self-hosted photo and video management solution.",
            "org.opencontainers.image.licenses": "AGPL-3.0",
            "org.opencontainers.image.revision": "8f69d9d0ece02d74e53bc34a9c430458d4fb87ad",
            "org.opencontainers.image.source": "https://github.com/immich-app/immich",
            "org.opencontainers.image.title": "immich",
            "org.opencontainers.image.url": "https://github.com/immich-app/immich",
            "org.opencontainers.image.version": "v1.118.0"
        },
        "OnBuild": null,
        "OpenStdin": false,
        "StdinOnce": false,
        "Tty": false,
        "User": "",
        "Volumes": null,
        "WorkingDir": "/usr/src/app"
    },
    "Created": "2025-05-09T02:55:13.326306366Z",
    "Driver": "btrfs",
    "ExecIDs": null,
    "GraphDriver": {
        "Data": null,
        "Name": "btrfs"
    },
    "HostConfig": {
        "AutoRemove": false,
        "Binds": null,
        "BlkioDeviceReadBps": null,
        "BlkioDeviceReadIOps": null,
        "BlkioDeviceWriteBps": null,
        "BlkioDeviceWriteIOps": null,
        "BlkioWeight": 0,
        "BlkioWeightDevice": null,
        "CapAdd": null,
        "CapDrop": null,
        "Cgroup": "",
        "CgroupParent": "",
        "CgroupnsMode": "host",
        "ConsoleSize": [
            0,
            0
        ],
        "ContainerIDFile": "",
        "CpuCount": 0,
        "CpuPercent": 0,
        "CpuPeriod": 0,
        "CpuQuota": 0,
        "CpuRealtimePeriod": 0,
        "CpuRealtimeRuntime": 0,
        "CpuShares": 0,
        "CpusetCpus": "",
        "CpusetMems": "",
        "DeviceCgroupRules": null,
        "DeviceRequests": null,
        "Devices": null,
        "Dns": null,
        "DnsOptions": null,
        "DnsSearch": null,
        "ExtraHosts": [],
        "GroupAdd": null,
        "IOMaximumBandwidth": 0,
        "IOMaximumIOps": 0,
        "IpcMode": "private",
        "Isolation": "",
        "Links": null,
        "LogConfig": {
            "Config": {},
            "Type": "db"
        },
        "MaskedPaths": [
            "/proc/asound",
            "/proc/acpi",
            "/proc/kcore",
            "/proc/keys",
            "/proc/latency_stats",
            "/proc/timer_list",
            "/proc/timer_stats",
            "/proc/sched_debug",
            "/proc/scsi",
            "/sys/firmware"
        ],
        "Memory": 0,
        "MemoryReservation": 0,
        "MemorySwap": 0,
        "MemorySwappiness": null,
        "Mounts": [
            {
                "Source": "immich_model-cache",
                "Target": "/cache",
                "Type": "volume",
                "VolumeOptions": {}
            }
        ],
        "NanoCpus": 0,
        "NetworkMode": "immich_default",
        "OomKillDisable": false,
        "OomScoreAdj": 0,
        "PidMode": "",
        "PidsLimit": null,
        "PortBindings": {},
        "Privileged": false,
        "PublishAllPorts": false,
        "ReadonlyPaths": [
            "/proc/bus",
            "/proc/fs",
            "/proc/irq",
            "/proc/sys",
            "/proc/sysrq-trigger"
        ],
        "ReadonlyRootfs": false,
        "RestartPolicy": {
            "MaximumRetryCount": 0,
            "Name": "always"
        },
        "Runtime": "runc",
        "SecurityOpt": null,
        "ShmSize": 67108864,
        "UTSMode": "",
        "Ulimits": null,
        "UsernsMode": "",
        "VolumeDriver": "",
        "VolumesFrom": null
    },
    "HostnamePath": "/volume3/@docker/containers/a63fa5dabefa35c83b1c0d9e0e6de487e8cbd6d25a8f3107f27234a46a9cd999/hostname",
    "HostsPath": "/volume3/@docker/containers/a63fa5dabefa35c83b1c0d9e0e6de487e8cbd6d25a8f3107f27234a46a9cd999/hosts",
    "Id": "a63fa5dabefa35c83b1c0d9e0e6de487e8cbd6d25a8f3107f27234a46a9cd999",
    "Image": "sha256:5aeea9d29ed8edf8ce9abb7cf5c4f51976c6e4842d78553607514f54941aba55",
    "LogPath": "/volume3/@docker/containers/a63fa5dabefa35c83b1c0d9e0e6de487e8cbd6d25a8f3107f27234a46a9cd999/log.db",
    "MountLabel": "",
    "Mounts": [
        {
            "Destination": "/cache",
            "Driver": "local",
            "Mode": "z",
            "Name": "immich_model-cache",
            "Propagation": "",
            "RW": true,
            "Source": "/volume3/@docker/volumes/immich_model-cache/_data",
            "Type": "volume"
        }
    ],
    "Name": "/immich_machine_learning",
    "NetworkSettings": {
        "Bridge": "",
        "EndpointID": "",
        "Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "HairpinMode": false,
        "IPAddress": "",
        "IPPrefixLen": 0,
        "IPv6Gateway": "",
        "LinkLocalIPv6Address": "",
        "LinkLocalIPv6PrefixLen": 0,
        "MacAddress": "",
        "Networks": {
            "immich_default": {
                "Aliases": [
                    "immich_machine_learning",
                    "immich-machine-learning",
                    "a63fa5dabefa"
                ],
                "DriverOpts": null,
                "EndpointID": "42dbc7d866804298b5e901a1042e8cb519425a47412a65d48b68385c8494f179",
                "Gateway": "192.168.16.1",
                "GlobalIPv6Address": "",
                "GlobalIPv6PrefixLen": 0,
                "IPAMConfig": null,
                "IPAddress": "192.168.16.3",
                "IPPrefixLen": 20,
                "IPv6Gateway": "",
                "Links": null,
                "MacAddress": "02:42:c0:a8:10:03",
                "NetworkID": "f1c19c6e05768d15c1c2b42b23f0c5a31c1e9c08043580ab238f89145b7e088e"
            }
        },
        "Ports": {},
        "SandboxID": "6372392feb3ff0c3dd6b6c6a7f988ca9c05d25d4645994ff4f6daf5e64884e4f",
        "SandboxKey": "/var/run/docker/netns/6372392feb3f",
        "SecondaryIPAddresses": null,
        "SecondaryIPv6Addresses": null
    },
    "Path": "tini",
    "Platform": "linux",
    "Portainer": {
        "ResourceControl": {
            "Id": 11,
            "ResourceId": "4_immich",
            "SubResourceIds": null,
            "Type": 6,
            "UserAccesses": [
                {
                    "UserId": 2,
                    "AccessLevel": 1
                }
            ],
            "TeamAccesses": [],
            "Public": false,
            "AdministratorsOnly": false,
            "System": false
        }
    },
    "ProcessLabel": "",
    "ResolvConfPath": "/volume3/@docker/containers/a63fa5dabefa35c83b1c0d9e0e6de487e8cbd6d25a8f3107f27234a46a9cd999/resolv.conf",
    "RestartCount": 0,
    "State": {
        "Dead": false,
        "Error": "",
        "ExitCode": 0,
        "FinishedAt": "0001-01-01T00:00:00Z",
        "Health": {
            "FailingStreak": 0,
            "Log": [
                {
                    "End": "2025-05-09T21:50:36.786967953+09:00",
                    "ExitCode": 0,
                    "Output": "",
                    "Start": "2025-05-09T21:50:35.899822723+09:00"
                },
                {
                    "End": "2025-05-09T21:51:07.712950836+09:00",
                    "ExitCode": 0,
                    "Output": "",
                    "Start": "2025-05-09T21:51:06.805488467+09:00"
                },
                {
                    "End": "2025-05-09T21:51:38.620372151+09:00",
                    "ExitCode": 0,
                    "Output": "",
                    "Start": "2025-05-09T21:51:37.733620377+09:00"
                },
                {
                    "End": "2025-05-09T21:52:09.528262523+09:00",
                    "ExitCode": 0,
                    "Output": "",
                    "Start": "2025-05-09T21:52:08.644939518+09:00"
                },
                {
                    "End": "2025-05-09T21:52:40.457659122+09:00",
                    "ExitCode": 0,
                    "Output": "",
                    "Start": "2025-05-09T21:52:39.54724088+09:00"
                }
            ],
            "Status": "healthy"
        },
        "OOMKilled": false,
        "Paused": false,
        "Pid": 3169,
        "Restarting": false,
        "Running": true,
        "StartedAt": "2025-05-09T02:55:14.603706795Z",
        "Status": "running"
    }
}

[bo0tzz]

"org.opencontainers.image.version": "v1.118.0"

Like I suggested above, you're running a pretty old version of the ML image. Updating it will fix this.

[reheinz]

very very Thank you. I don't know it. It works well!!!