Skip to content

oans v1.2.0

Choose a tag to compare

View all tags
@martinus martinus released this 18 Jul 19:44
v1.2.0
a3d0ab8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The first release carrying the fork's full feature set — 31 commits since 1.1.1. Everything is additive and backward-compatible; the CLI is a superset of duperemove's and hashfiles rebuild automatically.

✨ New features

  • --stats — inspect a hashfile: file/hash counts, duplication ratio, file-size summary, the top duplicate groups with an example path, and how much a VACUUM would reclaim.
  • Run history & metrics — every run is recorded in the hashfile. --history shows space reclaimed over time and a lifetime total; --json exports current metrics for a dashboard.
  • Self-describing hashfile — each run stores its options, paths and excludes, so oans --hashfile=FILE (no other arguments) replays the last run incrementally. Refuses rather than pruning if every stored path has vanished.
  • --autotune — benchmarks your backing storage (NAS/HDD/RAID) and picks the fastest --io-threads, persisting the winner in the hashfile.
  • systemd timer templates — make install-systemd, then systemctl enable --now oans@<name>.timer for weekly, idle-priority dedupe. See systemd/.
  • NAS quick-start guide — a complete walkthrough from first scan to scheduled, monitored dedupe: docs/nas-quickstart.md.

⚡ Performance

  • Skip already-shared files up front instead of re-reading them — warm re-runs on a mostly-stable tree are dramatically faster (a deduped 2M-file / 230 GiB tree rescans in ~92 s vs ~11 min for upstream 0.15.2).
  • No cross-generation reprocessing of duplicate groups — dedupe phase ~294 s → ~188 s (~36%), kernel dedupe traffic roughly halved.
  • Batched SQLite transactions (~10 s cadence) — ~24% faster rescans, file-lock syscalls cut from hundreds of thousands to a few hundred.
  • Parallel directory walk and a compact 64-bit path-hash index for large trees.

🎯 Reporting & correctness

  • Honest reclaimed-space reporting — the summary now shows the disk space actually freed (one physical copy kept per group), not the fiemap "net change in shared extents" that double-counted pairs. The stable net change in shared extents line is still emitted for scripts (piped / -q).
  • Safe concurrent report modes — --stats/--history/--json open the hashfile read-only, so they're safe to run while a scan or dedupe is in progress.
  • Hardlink safety fix — INSERT OR REPLACE could cascade-delete other hardlinks' rows and silently empty the hashfile; guarded, with a regression test.
  • Permission hint when a hashfile can't be opened; several dedupe-progress display fixes.

🧹 Housekeeping

  • Removed --fdupes and legacy/testing options (--hash-threads, -D, --read/--write-hashes); project-wide dead-code cleanup.
  • Reworked README leading with the value proposition; build-dependency install commands for Fedora and Debian/Ubuntu.

📦 Install

make && sudo make install          # oans + a duperemove compat symlink
sudo make install-systemd          # optional: the oans@ timer/service templates

Requires Linux 3.13+ on btrfs or XFS. Build deps and full details in the README.

Assets 4
Loading