Skip to content

oans v1.5.0

Choose a tag to compare

@martinus martinus released this 25 Jul 11:37
ccf8b81

Reaches files that were previously skipped, fixes two crashes, and puts the
whole codebase under ASAN, UBSAN and ThreadSanitizer in CI.

Features

  • Hash and deduplicate files whose absolute path exceeds PATH_MAX (#124,
    closes #117). Previously these were skipped with a warning. The kernel rejects
    any single pathname argument over 4096 bytes, so the new src/longpath.c
    reaches them by opening a reachable ancestor and openat-walking the rest,
    advancing by the longest run of components that fits one syscall argument — a
    4620-byte path costs 5 opens instead of 22. In-range paths take the plain
    open()/opendir()/stat() fast path internally, so the hot path is
    unchanged. No hashfile schema change and no DB_FILE_MINOR bump.
  • Hashing throughput during the scanning phase (#120). The progress display
    now shows live MB/s while hashing, not just during dedupe.

Fixes

  • Fixed a use-after-free in partial mode (#128, fixes #123) that caused
    intermittent crashes, plus three remaining PATH_MAX assumptions.
  • Fixed a use-after-free found by ThreadSanitizer (#129).
  • The version object now rebuilds when git describe changes (#122), so a
    binary can no longer report a stale version string.

Hardening

  • CI gained clang ASAN and UBSAN legs running both test suites (#125), and
    now fails on warnings with stricter flags plus dependabot (#127).
  • ThreadSanitizer is usable against GLib (#129), with a suppression that
    matches current distros (#131).
  • CI runs once per commit instead of twice (#130), and make check-all
    runs every CI leg in one command (#132).

All of this runs against a real btrfs and XFS scratch filesystem.

Tooling

  • scripts/bench-dedupe.py (#121), a reproducible larger-than-RAM dedupe
    benchmark harness; methodology is documented in docs/benchmarks.md.

Deduplication goes through the kernel's FIDEDUPERANGE ioctl, which
byte-compares every range before sharing it.