panic: index out of bounds

[wdecoster]

Hi,

Great to see Rust tools at ONT :)
I just downloaded the latest binary and ran the command below:

modkit pileup --partition-tag HP --combine-strands --cpg --ref ~/GRCh38.fa alignment.cram calls.bed --region chr10:13107989-13108319

First, I noted that --cpg and --ref were required, and also the name of an output file while I want to partition per haplotype. That seems weird, and I don't know what would have happened to my output if it didn't end up panicking.

modkit outputted
> parsing region chr10:13107989-13108319
and then took a suspiciously long time (2min45s) before running into:

thread 'main' panicked at 'index out of bounds: the len is 196 but the index is 18446744073709551615', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/rust-htslib-0.43.1/src/bam/mod.rs:847:41
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The index position looks like an underflow to me. The same error happened on another comparable interval. Please let me know what I can do to help debug this.

Wouter

[ArtRand]

Hello @wdecoster,

First off, the panic. The error is coming from a somewhat new part of rust-htslib (and newly utilized in modkit) where we look at the index to determine how many reads we can expect to sample from a region. Looks like if you use a BAM instead of CRAM you could avoid the code that's producing this error (ref). I know it's annoying, but maybe convert to BAM/BAI? I'll be sure to add more test coverage with CRAM input files, which is severely lacking right now.

As far as the options/flags, --cpg could be removed in the future for a more generic --motif option, but any time you're using a reference motif (CpG in this case), modkit requires the reference sequence. This limitation could be removed by parsing the MD tag, of course, and this is on the roadmap.

When using --partition-tag the output positional argument should be a directory, similar to the --bedgraph flag, the documentation will be updated to make this more clear.

Lastly, the program appeared to stall for nearly 3 minutes because when using CRAM the index stats method scans through all of the records. While it's better to know how many records are aligned to each contig when performing sampling, it's probably not worth this scan - older versions of modkit didn't use the index stats so modkit should fall back to the old strategy when using CRAM input. I'll put this on TODOs for the next release.

Could you let me know if switching to BAM/BAI solves the problem? Also if you could provide the full traceback that would be great. Thanks,

A

[ArtRand]

@wdecoster release 0.1.11 should solve this problem. I was able to reproduce the error by making a CRAM with aligned and unaligned records. I'll bubble it up to rust-htslib.

[wdecoster]

Awesome, thank you!