no space left on device - how to recover?

[mabod]

The bug

I just started to test out immich server on a debian vm and uploaded a couple of pictures. That was too many so that the server has no space left. I can not delete files from the web interface because there is no space left. How do I recover from that?

The OS that Immich Server is running on

debian 12

Version of Immich Server

1.120.2

Version of Immich Mobile App

none

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

╰─# cat docker-compose.yml
#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - '2283:2283'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:2ba50e1ac3a0ea17b736ce9db2b0a9f6f8b85d4c27d5f5accc6a416d8f42c6d5
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command:
      [
        'postgres',
        '-c',
        'shared_preload_libraries=vectors.so',
        '-c',
        'search_path="$$user", public, vectors',
        '-c',
        'logging_collector=on',
        '-c',
        'max_wal_size=2GB',
        '-c',
        'shared_buffers=512MB',
        '-c',
        'wal_compression=on',
      ]
    restart: always

volumes:
  model-cache:

Your .env content

╰─# cat .env
# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

uploading too many pictures so that no space is left

Relevant log output

No response

Additional information

No response

[bo0tzz]

Give the VM more disk space and grow the partition, or SSH in and delete some files manually to make space. This isn't an Immich issue.

[mabod]

This VM is my immich test environment to learn about the installation and how it behaves before I install it on my real server. Giving this VM more disc space it not the solution because this would not work on my real, physical server.

"Deleting some files" is not so easy. Which files should that be? I can not delete files belonging to the operating system. All other files belong to immich. The biggest folder is library/upload. Can I just delete files from there without messing up the immich database?

[bo0tzz]

No, you cannot just delete files from there. On a "production" server you need to avoid running out of disk space, and if you do run out of disk space you need to find something that you can delete or move to alternative storage temporarily while you add more disks.

Again, this is not an Immich issue, this is general system administration stuff.

[Eris76]

hey, I had the same issue and found this thread. it can also be replicated when installing Immich and pulling images

1- First run in console :df -h

2- Check if a Logical directory is almost full : in my case dev/mapper/ubuntu--lv was 90% full

3- run : sudo lvresize -L 60.00G /dev/(whatever directory you want to resize) to resize logical volume

4- run : sudo resize2fs /your directory 99999999 to increase the filesystem

it will result in an error saying : Filesystem max value is (example) 6 100 000

5 run again: sudo resize2fs /your directory 6100000 to increase the filesystem to the max

6- problem should be solved

In the example below I increased the logical volume to 60Go but you can pick whatever your host disk can allow.