Skip to content

midzelis/immich-dev

BranchesTags

Repository files navigation

immich-dev

Development and testing toolkit for Immich. Provides scripts for setting up isolated test environments, managing backups/restores, and running performance benchmarks.

Prerequisites

Required for all workflows:

  • Docker + Docker Compose
  • Python 3.10+ (auto-installed via ./dev wrapper)
  • sudo access (see Permissions below)

Test data:

  • You must provide your own photo archives as ingest/*.zip files

Permissions

Immich containers run as root, which means the library/ and postgres/ directories will be owned by root. The cleanup commands require sudo to delete these directories.

Your user must be a sudoer. To add your user to the sudo group:

# Debian/Ubuntu
sudo usermod -aG sudo $USER

# Fedora/RHEL/Arch
sudo usermod -aG wheel $USER

Log out and back in for the group change to take effect.

Concepts

sandbox

The sandbox/ directory represents bind-mount locations for Docker Compose. It contains:

  • library/ - Media files (photos, videos, thumbnails)
  • postgres/ - PostgreSQL database files
  • docker-compose.yml - Container definitions
  • .env - Environment configuration
  • .credentials - Generated API tokens

This same bind-mount pattern is used by both production and dev Immich deployments. The scripts support targeting either location.

Compose Files

The CLI supports targeting different docker-compose files:

Target Compose File Use Case
sandbox (default) sandbox/docker-compose.yml Isolated test environment
dev repo /path/to/immich/docker/docker-compose.dev.yml Development with source code
production /path/to/immich/docker/docker-compose.yml Production-like testing

Key insight: Backup and restore work consistently by operating inside containers:

  • Backup reads from container's /data mount
  • Restore extracts directly into container's filesystem
  • The host path (UPLOAD_LOCATION) is managed by docker-compose via .env

Override Mechanism

When benchmarking, docker-compose.bench.yml overrides volume mounts to combine:

  • Dev source code from BENCH_DEV_PATH
  • Test data from sandbox/library/
  • Isolated database in a named volume bench-postgres

This allows running your modified Immich code against a known dataset.

Quick Start

# Full automated setup (downloads, starts containers, creates admin, ingests photos, creates backups)
./dev seed

This will:

  1. Download/configure Docker Compose files
  2. Clean up any existing installation
  3. Start Immich containers
  4. Create admin user and API credentials
  5. Ingest photos from ingest/*.zip files
  6. Wait for all background jobs to complete
  7. Create backups in media-backups/

Access the running instance at http://localhost:3000

Default credentials: admin@example.com / password

Available Commands

./dev --help                    # Show all commands
./dev stop                      # Stop all Immich containers
./dev inspect                   # Start sandbox for manual inspection
./dev clean                     # Remove containers, volumes, and/or data
./dev backup                    # Create database and library backup
./dev restore                   # Restore from backups
./dev seed                      # Full automated setup
./dev bench                     # Benchmark tools (see below)

Typical Use Cases

Create a baseline snapshot

Capture the current state of your Immich instance as a reusable test baseline:

# Backup from running containers (live backup)
./dev backup --name baseline --no-stop

# Or stop services first for consistent backup
./dev backup --name baseline

Output: media-backups/baseline.database.sql.gz and media-backups/baseline.library.tar

Restore baseline to sandbox

Load your baseline into the isolated sandbox environment:

# Restore to sandbox (starts containers automatically)
./dev restore --name baseline

# Database only (skip library extraction)
./dev restore --name baseline --db-only

This starts fresh containers, extracts the library into the container, and restores the database.

Restore to running containers

Load test data into already-running containers:

# Restore to running containers (--no-start assumes they're already up)
./dev restore --name baseline --no-start

# Database only (if library is already in place)
./dev restore --name baseline --no-start --db-only

Clean and start fresh

# Interactive cleanup
./dev clean

# Remove everything without prompts
./dev clean --force

# Remove only containers and volumes (keep data)
./dev clean --containers --volumes

Database-only restore for benchmarking

Reset the database to a known state between benchmark runs without re-extracting the library:

./dev restore --db-only --no-start

Directory Structure

immich-dev/
├── dev                      # Main CLI wrapper (auto-installs Python environment)
├── cli/                     # Python CLI source code
│   ├── src/
│   │   ├── commands/        # Main commands (backup, restore, clean, etc.)
│   │   ├── bench/           # Benchmarking commands and core
│   │   └── utils/           # Shared utilities
│   └── pyproject.toml
├── sandbox/                 # Docker Compose environment
│   ├── docker-compose.yml
│   ├── .env
│   ├── .credentials         # Generated API tokens
│   ├── library/             # Uploaded media (generated)
│   └── postgres/            # Database files (generated)
├── bench-suites/            # Benchmark suite configurations
├── bench-results/           # Benchmark results (JSON)
├── ingest/                  # Place your photo zip files here
├── media-backups/           # Backup files
└── flake/                   # Nix development environment (optional)

Command Reference

backup

Create database and library backups from running containers.

./dev backup [OPTIONS]

Options:
  --mode [auto|docker|containerless]  # Runtime mode (auto-detect by default)
  --server-container TEXT             # Server container name (Docker mode only)
  --db-container TEXT                 # Postgres container name (Docker mode only)
  --db-username TEXT                  # Postgres username
  --output-dir PATH                   # Where to save backups (default: ./media-backups)
  --name TEXT                         # Backup name prefix (default: current date)
  --no-library                        # Skip library backup (database only)
  --db-only                           # Skip library backup (database only)
  --no-stop                           # Don't stop containers before backup

Examples:

./dev backup --name baseline           # Named backup
./dev backup --no-stop                 # Live backup (don't stop services)
./dev backup --db-only --name snapshot # Database only

restore

Restore Immich from backups in media-backups/.

./dev restore [OPTIONS]

Options:
  --mode [auto|docker|containerless]  # Runtime mode (auto-detect by default)
  --start/--no-start                  # Start containers before restore (default: start)
  --compose-file PATH                 # Compose file to start containers (default: sandbox/docker-compose.yml); only used when starting containers
  --db-container TEXT                 # Postgres container name
  --server-container TEXT             # Server container name
  --name TEXT                         # Backup name prefix (default: latest by date)
  --build-image                       # Build a data image from local backups
  --from-image TEXT                   # Restore from a data image instead of local files
  --db-only                           # Only restore database, skip library extraction

Examples:

./dev restore                              # Restore latest backup to sandbox (starts containers)
./dev restore --name 20250130              # Restore specific backup by name

# Already-running containers
./dev restore --no-start                   # Restore to running containers (validates they're up)
./dev restore --no-start --db-only         # Database only, skip library extraction

# Different compose files
./dev restore --compose-file /path/to/immich/docker/docker-compose.dev.yml   # Start dev containers and restore
./dev restore --compose-file /path/to/immich/docker/docker-compose.yml       # Start production containers and restore

# Benchmarking workflow
./dev restore                              # Initial restore with library
./dev restore --no-start --db-only         # Reset DB between runs (fast)

# From Docker image
./dev restore --from-image media-backups:baseline  # Restore from pre-built data image

clean

Remove Immich containers, volumes, and/or data.

./dev clean [OPTIONS]

Options:
  --compose-file PATH              # Docker compose file (default: sandbox/docker-compose.yml); .env beside it defines data paths
  --containers/--no-containers     # Remove containers
  --volumes/--no-volumes           # Remove Docker volumes
  --data/--no-data                 # Remove bind mount data (DESTRUCTIVE; reads UPLOAD_LOCATION from .env)
  --force, -f                      # Remove everything without prompts

Examples:

./dev clean                           # Interactive cleanup
./dev clean --force                   # Remove everything without prompts
./dev clean --containers --volumes    # Remove containers and volumes, keep data
./dev clean --compose-file /path/to/immich/docker/docker-compose.dev.yml  # Clean dev compose

seed

Full automated setup: download, cleanup, start, admin, ingest, backup.

./dev seed [OPTIONS]

Options:
  --skip-ingest                    # Skip photo ingestion step
  --from-step INTEGER [1-6]        # Start from a specific step (1-6)

Steps:

  1. Download/Setup
  2. Cleanup
  3. Start
  4. Admin
  5. Ingest
  6. Backup

stop

Stop all Immich containers.

./dev stop

inspect

Start sandbox containers for manual inspection.

./dev inspect

Access at http://localhost:3000 with admin@example.com / password.

Benchmarking

Fetch asset IDs

Before benchmarking, fetch asset IDs from the database:

./dev bench fetch-assets --output assets.txt -n 20000

Run a single benchmark

./dev bench run-get-thumbs --assets assets.txt --concurrency 6

Options for run-get-thumbs:

--api-key TEXT              # Immich API key (or set IMMICH_API_KEY)
--email TEXT                # Admin email for authentication (default: admin@example.com)
--password TEXT             # Admin password
--credentials-file PATH     # File to store/load API credentials
--base-url TEXT             # Immich server base URL (default: http://127.0.0.1:3000)
--assets PATH               # Path to file containing asset IDs (one per line)
--num-requests, -n INTEGER  # Number of requests to make (omit for all assets)
--concurrency, -c INTEGER   # Number of concurrent connections (default: 6)
--output, -o PATH           # Save results to JSON file
--label TEXT                # Label for this benchmark run
--group TEXT                # Group name for organizing results

Benchmark suites

Run multiple benchmark configurations automatically:

# Create a suite configuration
./dev bench suite --init my-experiment

# List available suites
./dev bench suite --list

# Dry run (show what would happen)
./dev bench suite my-experiment --dry-run

# Execute the suite
./dev bench suite my-experiment

Suite file format (bench-suites/my-experiment.yaml):

# Required: path to Immich source repo
dev: /path/to/immich

# Use dev compose instead of sandbox
use_sandbox: false

# Git tag/branch/jj bookmark (default for all series)
tag: main

# Asset IDs file
assets: assets.txt

# Concurrent connections (default for all series)
concurrency: 6

# Group name for organizing results
group: my-experiment

# Skip database restore after code switch (default: false)
skip_restore: false

# Backup name to restore between runs
backup: baseline

# Image size to retrieve: thumb or preview (default: thumb)
size: thumb

# Parameter sweep - what varies across all series
variants:
  - concurrency: 6
  - concurrency: 12
  - concurrency: 24

# Series - different conditions to compare
series:
  - name: main-branch
    label_prefix: main
    tag: main

  - name: feature-branch
    label_prefix: feature
    tag: my-feature-branch

# This creates 6 runs (2 series × 3 variants):
# - main 6t, main 12t, main 24t
# - feature 6t, feature 12t, feature 24t

Options for suite:

--init TEXT              # Create example configuration file
--list                   # List available suite files
--dry-run                # Show what would be run without executing
--continue-on-error      # Continue running remaining benchmarks if one fails
--base-url TEXT          # Immich server base URL

Compare results

Compare benchmark results from a group:

./dev bench compare --group my-experiment

Options:

--group, -g TEXT                          # Compare all results with this group name
--sort, -s [timestamp|label|throughput]   # Sort order for results
--output, -o [table|json]                 # Output format

Or compare specific files:

./dev bench compare result1.json result2.json result3.json

View results

View a single benchmark result:

./dev bench view <result-file.json>

List results

List all saved benchmark results:

./dev bench list

Evict ZFS ARC cache

Attempt to evict ZFS ARC cache (requires host access):

./dev bench evict-arc --timeout 5

Note: Cannot evict ARC from within LXC containers. For reliable cold benchmarks, use a ZFS dataset with primarycache=none.

Cold Benchmarks in LXC (ZFS)

When running in an LXC container on a ZFS host, the host's ZFS ARC (RAM cache) cannot be cleared from within the container. For accurate "cold cache" benchmarks:

1. Create an uncached ZFS dataset (on host)

# On host - creates a ZFS dataset (filesystem)
zfs create rpool/immich-bench-data \
    -o primarycache=none \
    -o secondarycache=none \
    -o compression=lz4 \
    -o mountpoint=/mnt/immich-bench-data
  • primarycache=none - Bypasses ARC (RAM cache). Data is never cached.
  • secondarycache=none - Bypasses L2ARC (SSD cache, if configured).

2. Add bind mount to LXC config

Standard LXC format (add to container config):

lxc.mount.entry: /mnt/immich-bench-data mnt/bench-data none rbind,create=dir 0 0

Or Proxmox shorthand (/etc/pve/lxc/<id>.conf):

mp0: /mnt/immich-bench-data,mp=/mnt/bench-data

Then restart the container.

3. Use the uncached dataset for benchmark media

# Option A: Symlink
rm -rf sandbox/library
ln -s /mnt/bench-data/library sandbox/library

# Option B: Update .env
# UPLOAD_LOCATION=/mnt/bench-data/library

Configuration Files

sandbox/.env

UPLOAD_LOCATION=./library      # Media storage path
DB_DATA_LOCATION=./postgres    # Database storage path
IMMICH_VERSION=v2              # Container version (or pin to specific)
DB_PASSWORD=postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

sandbox/.credentials

Auto-generated during ./dev seed or when running commands that need authentication:

ACCESS_TOKEN='...'
API_KEY='...'
ADMIN_EMAIL='admin@example.com'
ADMIN_PASSWORD='password'

Test Data

You must provide your own photo archives:

  1. Place zip files in the ingest/ directory
  2. During ./dev seed, photos are extracted and uploaded
  3. GPS metadata is automatically added to photos lacking it (via exiftool)

Development

The ./dev wrapper automatically sets up the Python environment on first run using uv or pip. No manual installation required.

Shell completion:

# Bash
source ./cli/completions/dev.bash

# Zsh
source ./cli/completions/dev.zsh

Development commands:

./dev lint         # Run ruff linter
./dev format       # Format code with ruff

About

Immich development tools, and automated benchmark suite

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages