Skip to content

cf1-rs v0.6.0

Choose a tag to compare

@rob-p rob-p released this 30 Oct 05:13
· 13 commits to main since this release
3836184

Behavior change (output):

  • Unitig ids change. The MPHF is now sux 0.15's PHastR (in place of
    ph's FMPH), so the MPHF indices that name unitigs, and hence the ids in
    .cf_seg/.cf_seq, differ from 0.5.x. The compacted de Bruijn graph itself
    is identical (same unitig sequences and tilings, up to renaming). Output is
    now also independent of the MPHF construction regime (in-memory vs.
    streaming) and of the thread count. Anything that compared cf1-rs output
    across versions by unitig id must re-baseline.

Breaking (library API; all modules are pub):

  • classify::Classifier::process_substring takes an extra
    seen: Option<&mut SeenWindows> argument.
  • directed_kmer::AnnotatedKmer has a new public field hash (its MPHF
    index); AnnotatedKmer::roll_to_next_kmer is removed in favor of the new
    batched directed_kmer::AnnotatedRun.
  • The ph dependency is dropped. cf_build and its options are unchanged.

Performance:

  • MPHF: PHastR in both regimes. In memory it is built straight from the key
    slice (k-mers no longer touch disk); the streaming regime feeds the keys once
    into PHastR's offline construction, deduplicating later bins in the
    background, so the dedup-file write and consolidation copy are gone.
  • Phases 4 and 5 resolve MPHF lookups in batches of up to 32 k-mers with state
    prefetching (x86-64 and aarch64); phase 4 skips content-defined windows a
    task has already applied.
  • Phases 1 and 2 batch small sequences (~4 Mbp per task, task-local
    histograms, fewer bin flushes); routing drops exact repeat super k-mers
    within a task. In-memory keys are gathered a wave of bins at a time into one
    pre-reserved vector.
  • GENCODE v50, 16 threads: P1 1.95 → 0.41 s, P2 2.57 → 0.95 s, P4 10.2 → 1.6 s,
    P5 8.7 → 3.0 s. GRCh38 genome (2.5B k-mers, streaming), 16 threads:
    110.7 → 76.2 s wall, peak RSS unchanged (~5.8 GB). Peak RSS of in-memory
    (transcriptome-scale) builds grows modestly (GENCODE v50: 3.8 → 4.5 GB).