umi tools not working

[anoronh4]

Description of bug

the new umitools module isn't processing the log as expected. it is finding the correct file but cannot parse it as expected. i ran the umitools command as follows:

PYTHONHASHSEED=0 umi_tools \
    dedup \
    -I M18-39155_T1.Aligned.sortedByCoord.out.bam \
    -S M18-39155_T1.bam \
    --output-stats M18-39155_T1 \
    --paired \
    --log M18-39155_T1.umitools_dedup.log
    --random-seed=100

i wonder if the original module was designed around single-end data, and needs some handling for paired end data.

File that triggers the error

M18-39155_T1.umitools_dedup.log

MultiQC Error log

$ multiqc . 
  /// MultiQC 🔍 | v1.14
...
╭───────────────────────────────── Oops! The 'umitools' MultiQC module broke... ─────────────────────────────────╮
│ Please copy this log and report it at https://github.com/ewels/MultiQC/issues                                  │
│ Please attach a file that triggers the error. The last file found was: ./M18-39155_T1.umitools_dedup.log       │
│                                                                                                                │
│ Traceback (most recent call last):                                                                             │
│   File "/usr/local/lib/python3.11/site-packages/multiqc/multiqc.py", line 654, in run                          │
│     output = mod()                                                                                             │
│              ^^^^^                                                                                             │
│   File "/usr/local/lib/python3.11/site-packages/multiqc/modules/umitools/umitools.py", line 36, in __init__    │
│     input_fname, data = self.parse_logs(f["f"])                                                                │
│                         ^^^^^^^^^^^^^^^^^^^^^^^                                                                │
│   File "/usr/local/lib/python3.11/site-packages/multiqc/modules/umitools/umitools.py", line 123, in parse_logs │
│     logdata[key] = value[1](line.partition(value[0])[2])                                                       │
│                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                       │
│ ValueError: invalid literal for int() with base 10: '5925335, Read pairs: 5925335, Read 2 unmapped: 7'         │
│                                                                                                                │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

[ewels]

Thanks for reporting @anoronh4! I fixed the bug and ended up refactoring the module code to use regular expressions. Hopefully it'll be more resistant to minor variations in the log output format now.

[mkyriak]

I wasn't sure if I should open a new issue, but I am getting the same error. I have also used umi-tools for paired-end data and not for single cell.

 /// MultiQC 🔍 | v1.14

|           multiqc | Search path : /scratch/mkyriakidou/Frontier/dedup/stats/umi_analysis/QC
|         searching | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 945/945  
|            picard | Found 135 AlignmentSummaryMetrics reports
|            picard | Found 135 HsMetrics reports
╭────────────────────────────────────────── Oops! The 'umitools' MultiQC module broke... ──────────────────────────────────────────╮
│ Please copy this log and report it at https://github.com/ewels/MultiQC/issues                                                    │
│ Please attach a file that triggers the error. The last file found was: ./FRONTIER_14160_Brain_VI_S6_dedup.log                    │
│                                                                                                                                  │
│ Traceback (most recent call last):                                                                                               │
│   File "/home/mkyriakidou/mambaforge/lib/python3.10/site-packages/multiqc/multiqc.py", line 654, in run                          │
│     output = mod()                                                                                                               │
│   File "/home/mkyriakidou/mambaforge/lib/python3.10/site-packages/multiqc/modules/umitools/umitools.py", line 36, in __init__    │
│     input_fname, data = self.parse_logs(f["f"])                                                                                  │
│   File "/home/mkyriakidou/mambaforge/lib/python3.10/site-packages/multiqc/modules/umitools/umitools.py", line 123, in parse_logs │
│     logdata[key] = value[1](line.partition(value[0])[2])                                                                         │
│ ValueError: invalid literal for int() with base 10: '8394804, Read pairs: 8394804, Chimeric read pair: 59349'                    │
│                                                                                                                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

[ewels]

I don't think I've pushed a release since I fixed this bug sadly, so you'll need to run the development version.

[mkyriak]

Thank you, I will do that.