Skip to content

v1.6.6 — Table-based UCE evidence and Apple silicon tuning

Choose a tag to compare

@GUIBA-EX GUIBA-EX released this 11 Oct 17:06
· 16 commits to master since this release

Highlights

  • Replaces FM-index extension in UCEFilter exact-match evidence with an exact run-k table when run-k ≤ 32 (default 19): each run-k window costs one lookup, and a MEM extends by comparing 32 packed bases per step against each occurrence of its run-k prefix. Larger run-k keeps the FM index.
  • Stores the run-k table compactly (two bits per reference base, one fingerprinted u32 per hash slot); peak memory on the benchmark below is about 16 MB above the FM index.
  • Detects the host processor and memory: -p auto reports performance and efficiency cores on Apple silicon, macOS measures available memory instead of falling back to a 512 MiB UCEFilter fragment bank, and compute threads ask macOS to prefer performance cores.
  • Assembler k-mer counting workers claim small blocks instead of fixed equal slices, so faster cores take more work.
  • Leaves scheduling unchanged: stage thread budgets, stage priorities and sample order still follow the GeneMiner2 dispatcher.

Performance

575,435-pair UCE benchmark used for v1.6.5 (2,211 bivalve loci; on-target pairs simulated from real DK40 UCE contigs, off-target pairs from a real WGS library; Apple M2, -p 8, default filter assemble --assembly-mode uce), two runs each:

Stage v1.6.5 v1.6.6
Total 47.3–49.8 s 32.3–33.2 s
First-pass filter 4.8–5.0 s 1.8–2.0 s
Rescue-round filter 18.8–20.0 s 5.5–6.7 s

Assembly time is unchanged. UCEFilter alone on a 675,435-pair variant of the benchmark: exact-match evidence 4.7 s → 0.85 s, peak RSS 331 MB → 347 MB.

Verification

  • All result files were byte-identical to v1.6.5 on the benchmark above (5,556 files) and on real phyluce Gallus UCE reads (5,123 files); only run logs and manifests differ.
  • New randomized equivalence tests compare the run-k table path with the FM index (seeds, occurrence counts, strand masks, MEM statistics), packed with bytewise prefix comparison, and k-mer counts across worker counts.
  • 286 workspace tests passed; cargo fmt --check and Clippy with warnings denied passed; release build completed.