[Feature] CLI: Single-pass upload (hash + dedup + upload) instead of reading files twice

[abo-chen]

I have searched the existing feature requests, both open and closed, to make sure this is not a duplicate request.

• Yes

The feature

Currently, the immich upload CLI reads all files twice during a single upload session:

1. Hashing files — reads every file to compute perceptual hashes
2. Uploading assets — reads non-duplicate files again to upload

For large libraries (e.g., 1TB+ with 100k+ files), this effectively doubles the I/O time. On my setup, hashing 1TB takes ~5 hours, and the upload adds another significant chunk of time — much of which is re-reading the same files from disk.

Proposed solution:

Merge the three phases into a single streaming pipeline:

For each batch of N files:

1. Hash files
2. Send batch hashes to server for dedup check
3. Upload only non-duplicate files
4. Move to next batch

This way each file is read at most once (hash + upload in the same pass), or twice only for non-duplicates that need uploading.

Benefits:

• ~50% reduction in disk I/O for large imports
• Faster total import time, especially for local/server deployments where network isn't the bottleneck
• Memory usage stays similar (batch-based processing)

Environment:

• Immich CLI v2.7.5
• Linux (local deployment)

Platform

• Server
• Web
• Mobile