oans v1.5.0
Reaches files that were previously skipped, fixes two crashes, and puts the
whole codebase under ASAN, UBSAN and ThreadSanitizer in CI.
Features
- Hash and deduplicate files whose absolute path exceeds
PATH_MAX(#124,
closes #117). Previously these were skipped with a warning. The kernel rejects
any single pathname argument over 4096 bytes, so the newsrc/longpath.c
reaches them by opening a reachable ancestor andopenat-walking the rest,
advancing by the longest run of components that fits one syscall argument — a
4620-byte path costs 5 opens instead of 22. In-range paths take the plain
open()/opendir()/stat()fast path internally, so the hot path is
unchanged. No hashfile schema change and noDB_FILE_MINORbump. - Hashing throughput during the scanning phase (#120). The progress display
now shows live MB/s while hashing, not just during dedupe.
Fixes
- Fixed a use-after-free in partial mode (#128, fixes #123) that caused
intermittent crashes, plus three remainingPATH_MAXassumptions. - Fixed a use-after-free found by ThreadSanitizer (#129).
- The version object now rebuilds when
git describechanges (#122), so a
binary can no longer report a stale version string.
Hardening
- CI gained clang ASAN and UBSAN legs running both test suites (#125), and
now fails on warnings with stricter flags plus dependabot (#127). - ThreadSanitizer is usable against GLib (#129), with a suppression that
matches current distros (#131). - CI runs once per commit instead of twice (#130), and
make check-all
runs every CI leg in one command (#132).
All of this runs against a real btrfs and XFS scratch filesystem.
Tooling
scripts/bench-dedupe.py(#121), a reproducible larger-than-RAM dedupe
benchmark harness; methodology is documented indocs/benchmarks.md.
Deduplication goes through the kernel's FIDEDUPERANGE ioctl, which
byte-compares every range before sharing it.