v2.59.0 — dupes default ignores + token cache

Highlights

• fallow dupes now ignores generated framework output by default. **/.next/**, **/.nuxt/**, **/.svelte-kit/**, **/.turbo/**, **/.parcel-cache/**, **/.vite/**, **/.cache/**, **/out/**, and **/storybook-static/** are dropped before tokenization. Authored-looking lib/, legacy/, and nested build/ directories stay in scope. Set duplicates.ignoreDefaults: false to opt out.
• Persistent token cache for duplication on large monorepos. Projects above duplicates.minCorpusSizeForTokenCache (default 5000 source files) reuse tokenized output across runs in .fallow/cache/dupes-tokens-vN/.
• Shingle prefilter for focused-mode duplication. When --changed-since runs against a project at or above duplicates.minCorpusSizeForShingleFilter (default 1024 files), unchanged files whose shingles do not overlap a changed file are dropped before suffix-array construction.
• Stack overflow on microsoft/TypeScript-scale projects fixed. Rayon worker stack is now pinned to 16 MiB across CLI and NAPI, so dead-code and health no longer abort with fatal runtime error: stack overflow on very deep trees.

Thanks @OmerGronich for the deep profiling on MUI (#243) and the stack-overflow report on microsoft/TypeScript (#247).

Added

• duplicates.ignoreDefaults config. Boolean (default true) controlling whether the new built-in framework-output ignores merge with duplicates.ignore.
• --explain-skipped global flag. Expands the human/markdown skipped-default-ignores note into per-pattern counts. Machine formats (JSON, SARIF, CodeClimate, compact) suppress the note.
• fallow init scaffolds a commented-out [duplicates] block. The generated .fallowrc.json is now valid JSONC end-to-end; both the JSON and TOML variants ship with example ignore additions for lib/, legacy/, __generated__/, and generated/.

Changed

• fallow dupes skipped-file note. Human and markdown output show a one-line summary of files skipped by the default ignores; pass --explain-skipped for per-pattern counts.
• --changed-since wires straight into the focused fast path. Resolving the changed-file set up front lets the engine engage the shingle prefilter and extraction-time interval pruning instead of running a full-corpus scan followed by a redundant post-filter. Also fixes a latent IntervalIndex coalescing bug in post-LCP filtering that was fragmenting intervals and keeping more groups than necessary.
• fallow audit --gate all skips the base-snapshot pass. Attribution is irrelevant to the verdict in --gate all mode, so the second analysis pass is dropped.
• Shared rayon global pool config across CLI and NAPI entry points. Both bindings now agree on stack-size and thread-count defaults so embedders see consistent parallel behaviour.

Fixed

• Rayon worker stack pinned to 16 MiB. Deep AST visitor and graph traversals could overflow Rust's default 8 MiB worker stack on very large real-world projects (e.g. microsoft/TypeScript), aborting dead-code and health analysis with fatal runtime error: stack overflow. The previous RUST_MIN_STACK=16777216 workaround is no longer needed. (Closes #247)

Upgrade notes

If your duplication percentage drops on upgrade, it is because fallow dupes now excludes generated framework output (the patterns above) by default. To restore the previous behaviour, add this to your config:

{
  "duplicates": {
    "ignoreDefaults": false
  }
}

Or merge specific framework directories back in via duplicates.ignore.

Full Changelog: v2.58.0...v2.59.0