oans v1.1.0
oans 1.1.0 builds on 1.0.0 with automatic hashfile maintenance, a correctness fix, and a distinct on-disk identity.
⚠️ One-time hashfile rebuild on upgrade
The hashfile format moves to 5.0 and is now branded with an oans application_id. The first oans run after upgrading will print Recreating hashfile .. and do one full re-scan to rebuild it. This is expected and only happens once; the hashfile is just a cache.
Highlights
- Automatic deleted-file pruning. Files removed from disk are now dropped from the hashfile automatically on the next scan (their rows and cascaded hashes), so a long-lived hashfile no longer grows without bound and the dedupe phase stops loading phantom groups for gone files. Freed space is reclaimed (
VACUUM) once enough of the file is unused. It's existence-based (stat), so scanning a subdirectory or sharing one hashfile across trees never prunes files that still exist. A "seen-set" built during the scan means a no-op rescan does zero extrastat()s — no steady-state cost. - Distinct hashfile identity. Every hashfile is stamped with a SQLite
application_id("oans") and the format is bumped to 5.0 (a clean break from duperemove's 4.x line). oans strictly refuses any hashfile that isn't branded as its own, so oans and duperemove can never mis-read each other's files. - Correctness fix. Fixed an uninitialised-memory read when loading the hashfile config UUID (
uuid_parseon an unterminated buffer). Added a valgrind suppressions file so the whole scan/dedupe/prune path is verifiably clean.
Notes
Benchmarks and behavior otherwise carry over from 1.0.0. On compressed filesystems, judge reclaimed space with compsize Disk Usage, not the logical "Deduplicated" figure. Original duperemove by Mark Fasheh and contributors; licensed GPLv2.