v9.3.0

Note

9.3.0 is available on 🐍 PyPI and 🐙 GitHub.

• Breaking: a dry run now says DRY RUN: N mails would be acted upon, but none will be altered. once, in place of a DRY RUN: Skip action. line per mail. Scripts matching the old wording need updating.
• Breaking: Python importers must switch to Strategy.function/Strategy.apply(), Action.perform() and DuplicateSet.select(), which replace strategy_function/apply_strategy(), perform_action() and categorize_candidates(). The CLI is unchanged.
• Add a hardlink-discarded action: each discarded mail becomes a hardlink to the copy kept in its set, reclaiming its space while staying in its own folder. Only byte-identical copies are linked, unless --hardlink-differing is passed. Closes #164.
• Add --cache and --cache-path, off by default, to reuse the hashes of previous runs: a second run over a 20,000-mail maildir takes half the time. Entries expire with their mail or when the hashing options change. Closes #87.
• Spread hashing and selection over worker processes instead of threads, so --jobs finally pays off: a first run over a 20,000-mail maildir drops from 3.4s to 1.8s at --jobs=4. Only folder-based boxes (maildir, MH, eml) fan out.
• Speed up hashing by about 2x and halve the file-system calls a maildir run makes per mail, taking a whole run about 2.6x faster with the default --hash-body skip. Refs #87.
• Cut the memory retained per mail from about 950 to 600 bytes. Refs #87.
• Settle the size and content thresholds over the whole duplicate set instead of comparing every pair: a single set of 200 copies took 1.90s and now takes 0.04s.
• Cut what a run prints by seven eighths, from 32,000 lines to 4,000 on a 20,000-mail corpus: mails and duplicate sets are now named at --verbosity DEBUG only. The selection step gained a progress bar.
• Add platformdirs to the runtime dependencies, which resolves the per-platform location of the hash cache.
• Fix the documentation's broken links: pipx's installation guide moved, and the ctime note now cites Python's os.path.getctime docs. Closes #899.
• Add a performance page covering where a run spends its time, the memory it holds and how to manage the hash cache, plus tutorial and design sections on hardlinking. Correct the per-mail memory figures, which claimed a few hundred bytes.

Full changelog: v9.2.0...v9.3.0