RuntimeWarning: invalid value encountered in subtract

[jamigo]

Description of bug

Last MultiQC v1.20 update came with the need to update plotly to v5.18. It ran fine on a couple of BWA+GATK runs, but it fails on a DRAGEN run. Unfortunately, the process doesn't really break, therefore no proper MultiQC debugging information is genereated, and we don't know which module breaks and which file is triggering the error. We do see this warnings through MultiQC output that we hope you'd be able to help us understand:

/mnt/netapp2/Store_JAL/.local/lib/python3.9/site-packages/numpy/core/_methods.py:173: RuntimeWarning: invalid value encountered in subtract
  x = asanyarray(arr - arrmean)
/mnt/netapp2/Store_JAL/.local/lib/python3.9/site-packages/multiqc/plots/plotly/violin.py:548: RuntimeWarning: invalid value encountered in subtract
  z_scores = np.abs((values - mean) / std_dev)

The html file is generated, but it hangs at "Loading report" and no mosdepth graphs are generated. In the past, we reported another mosdepth issue with MultiQC v1.16, but v1.18 did already solve it. Could the numpy v1.26.4 be the problem? Any other ideas?

File that triggers the error

No response

MultiQC Error log

$ multiqc .

  /// MultiQC 🔍 | v1.20

|           multiqc | Excluding modules 'gatk'
|           multiqc | Search path : /mnt/netapp2/Store_JAL/ngs/DRAGEN/test
|         searching | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 7322/7322
|            picard | Found 120 HsMetrics reports
|          mosdepth | Found reports for 120 samples
/mnt/netapp2/Store_JAL/.local/lib/python3.9/site-packages/numpy/core/_methods.py:173: RuntimeWarning: invalid value encountered in subtract
  x = asanyarray(arr - arrmean)
/mnt/netapp2/Store_JAL/.local/lib/python3.9/site-packages/multiqc/plots/plotly/violin.py:548: RuntimeWarning: invalid value encountered in subtract
  z_scores = np.abs((values - mean) / std_dev)
|            dragen | Found 240 reports
|            dragen | Found 120 reports
|     dragen_fastqc | Found 120 reports
|        bclconvert | 4 lanes and 120 samples found
|           multiqc | Report      : multiqc_report.html
|           multiqc | Data        : multiqc_data
|           multiqc | MultiQC complete

Before submitting

• I have read the troubleshooting documentation.
• I am using the latest release of MultiQC.
• I have included a full MultiQC log, not truncated.
• I have attached an input file (.zip if necessary) that triggers the error.

[vladsavelyev]

Thanks @jamigo for reporting the issue! Unfortunately, I can't figure out what might be the problem here without a reproducible example. Would you be able to share the DRAGEN QC outputs by any chance?

[jamigo]

By adding files one by one to MultiQC processing folder, I've traced down the error (or, at least, one of them) to a DRAGEN's sample.wgs_coverage_metrics.csv file.

In fact, I think I've found the error: if I change "inf" in the "Mean/Median" ratio section to 0.5 or any other number, everything works fine. Is there any problem handling "inf" values?

COVERAGE SUMMARY,,Mean/Median autosomal coverage ratio over genome,0.5
for instance, instead of
COVERAGE SUMMARY,,Mean/Median autosomal coverage ratio over genome,inf

If there are any others, I would be able to go on tracing issues once you suggest how to solve this one.

[vladsavelyev]

@jamigo, I see now, it's indeed the inf value that fails here. Python's float("inf") successfully parses it as a number, but in JavaScript, loading JSON with value: Infinity doesn't work. Fixed now with #2354

Thanks again for reporting this, and thanks for narrowing down the problem! Really appreciated. Also added this example into the test bundle.