Ditto

Duplicate file finder with a web UI. Scan folders, hash files, and browse duplicate groups by content (hash) or by inode (hardlinks). Designed to run on a Synology NAS via Docker.

Quick start

Local (Go)

# Clone and run
go run ./cmd/ditto
# Web UI at http://localhost:8080

To use a different port:

DITTO_PORT=3000 go run ./cmd/ditto

Docker

Images are published to GitHub Container Registry. Use the latest release or latest:

# Pull and run (use a version tag for production, e.g. ghcr.io/eargollo/ditto:v0.1.0)
docker run --rm -v ditto-data:/data -p 8080:8080 ghcr.io/eargollo/ditto:latest
# Web UI at http://localhost:8080

Scan roots you add in the UI must be paths inside the container. To scan a host folder, mount it and use the container path as the scan root:

docker run --rm \
  -v ditto-data:/data \
  -v /path/on/host/Photos:/scan/Photos:ro \
  -p 8080:8080 \
  ghcr.io/eargollo/ditto:latest
# In the UI, add scan root: /scan/Photos

To build from source instead: docker build -t ditto . then use the ditto image in the commands above.

Docker Compose

See docker-compose.yml. Set the image to ghcr.io/eargollo/ditto:latest (or a version tag), uncomment and set the volume for the folder(s) you want to scan, then:

docker compose up -d

Running on Synology

1. Install Container Manager (Package Center) if needed.
2. Get your user UID/GID — SSH into the NAS and run id admin (or your user). Example: uid=1026(admin) gid=100(users) → use PUID=1026, PGID=100.
3. Create a folder for Ditto data (e.g. Docker/ditto/data) in File Station. It will be owned by your user.
4. Create a container in Container Manager:
   • Image: ghcr.io/eargollo/ditto:latest (or a version tag like ghcr.io/eargollo/ditto:v0.1.0).
   • Environment: DATABASE_URL=..., DITTO_PORT=8080, PUID=<your UID>, PGID=<your GID>.
   • Volume (optional): only if you mount host folders to scan; no app data volume needed (state is in Postgres).
   • Optional — folders to scan: mount shared folders so they appear inside the container (e.g. Photos → /scan/Photos). In the UI, add scan root /scan/Photos (the path inside the container).
   • Port: map container port 8080 to a host port (e.g. 8080 or 32480).
5. Start the container and open http://<NAS-IP>:<host-port> in a browser.

Compose example (Synology): use host paths and set PUID/PGID to your DSM user (from id admin). Replace /volume1/... with your NAS paths.

services:
  ditto:
    image: ghcr.io/eargollo/ditto:latest
    container_name: ditto
    restart: unless-stopped
    environment:
      DITTO_PORT: 8080
      PUID: 1026   # your UID from "id admin"
      PGID: 100    # your GID
    volumes:
      - /volume1/Photos:/scan/Photos:ro
    ports:
      - "8080:8080"

In the UI, add scan root /scan/Photos. For more detail (permissions, troubleshooting), see Running on Synology.

Configuration

Variable	Default	Description
DATABASE_URL	(required)	PostgreSQL connection URL. All state is stored in Postgres.
DITTO_PORT	8080	HTTP port for the web UI.
PUID / PGID	1000 / 1000	(Docker only) Run the app as this user. On Synology, set to your DSM user's UID/GID when you mount host folders to scan. Use id youruser on the NAS to get the values.

Reference mode (validating correctness)

You can run a serial, in-memory pass over a directory (no database) that produces the same CSV format as the service export. Use it to validate the full pipeline on real data (e.g. on Synology):

1. Run the reference on the same root you will scan in the UI:

   ./ditto reference -o ref.csv /volume1/video

2. Start the service, add that root, run a scan and hash, then download the scan CSV from the scan details page.
3. Compare the two CSVs (e.g. sort both by path and diff). They should match (path, hash, size).

Reference uses the same exclude rules as the service and only hashes duplicate candidates (files whose size appears at least twice), so the CSV matches the service export. Inode reuse (hardlinks) is applied. No DATABASE_URL or server is required.

./ditto reference [-o output.csv] [-stats stats.csv] <root>
# -o      write CSV to file (default: stdout)
# -stats  write stats CSV (file_count, hashed_file_count, walk_duration_sec, etc.) for comparing reference vs service speed

Documentation

• Running on Synology — Deploy Ditto on a Synology NAS with Container Manager (Docker): image, volumes, env, and scan roots.
• Release process — How to build and publish the Docker image and optional binaries (versioning, multi-platform build, git tags).

Development

Tests and the app require PostgreSQL. Start the dev database, then run tests or the app:

docker compose -f docker-compose.dev.yml up -d
make test   # or: go test -p 1 ./...
export DATABASE_URL="postgres://ditto:ditto@localhost:5432/ditto?sslmode=disable"
make build  # or: go build -o ditto ./cmd/ditto
go run ./cmd/ditto   # or run ./ditto

Important: Tests must run with -p 1 (one package at a time) because they share a single Postgres instance and truncate the same tables; running go test ./... without -p 1 causes deadlocks. Use make test to get the correct flags.

Tests default to postgres://ditto:ditto@localhost:5432/ditto?sslmode=disable when DATABASE_URL is unset. The app still requires DATABASE_URL. See docker-compose.dev.yml for credentials and port.

License

See LICENSE if present.