Skip to content

v2.6.8: preserve coordinate sort order in streaming pipeline

Choose a tag to compare

@mtcicero26 mtcicero26 released this 16 Apr 18:43
· 135 commits to main since this release

⚠️ Reverse-strand MM/ML parsing bug — UPGRADE TO v2.9.0

Versions 2.6 – 2.8.2 had a parser bug that shifted m6A/m5C modification positions on reverse-aligned reads (~24 bp median), causing ns/nl/as/al/MA/AQ tags to be wrong on reverse reads. Forward reads unaffected. DAF with IUPAC (R/Y) encoding unaffected.

Upgrade: pip install --upgrade fiberhmm (v2.9.0+). See v2.9.0 release notes for details.


Bugfix release.

The streaming pipeline (used by `fiberhmm-run` and `fiberhmm-apply`) was writing skipped reads (`unmapped`, `low_mapq`, `too_short`, `no_modifications`, `extraction_failed`, etc.) immediately to the output BAM the moment they hit a filter, while processed reads were buffered in chunks and written out only after the worker pool finished the chunk.

The result: skipped reads landed at their stream-order position; processed reads landed delayed by one chunk. On coordinate-sorted input, this produced ~0.04% out-of-monotonic-order reads in the output — enough that `samtools index` rejected the file and forced a full `samtools sort` pass.

On large BAMs (e.g. a 360 GB Hia5 PacBio dataset reported in the field), that sort pass was running ~5 hrs and dominating the wall-clock budget.

Fix

Buffer skipped reads into `chunk_read_objs` alongside processed reads with a parallel `chunk_skip_flags` list. The drain step walks the chunk in original stream order — writing skipped reads as-is at their original position and applying worker results to the rest. Coordinate-sorted input → coordinate-sorted output → `samtools index` works directly, no sort pass needed.

Expected impact

  • Output BAMs from `fiberhmm-run` / `fiberhmm-apply` are now strictly coordinate-sorted when the input was.
  • You can drop the trailing `samtools sort` from your pipeline and `samtools index` directly.
  • On a 360 GB BAM, that's a multi-hour saving per sample.

Install

```bash
pip install --upgrade fiberhmm
```

🤖 Generated with Claude Code