data_format: "yaml" not working when provided via multiqc_config.yaml

[drpatelh]

Description of bug

Providing data_format: "yaml" via the standard multiqc_config.yaml is not working. However, providing the equivalent option via the CLI -k yaml is working fine. This could be a config parsing issue. This wasn't an issue in previous stable releases because we have been using this option in nf-core/viralrecon and it has worked.

Steps to reproduce

• Download a standard FastQC report to test with:

wget https://nf-core-awsmegatests.s3-eu-west-1.amazonaws.com/rnaseq/results-89bf536ce4faa98b4d50a8ec0a0343780bc62e0a/aligner_star_salmon/fastqc/GM12878_REP1_2_fastqc.zip

• Dowload and run latest Biocontainer to execute MultiQC

docker run -it --volume $HOME:$HOME --workdir $PWD quay.io/biocontainers/multiqc:1.13a--pyhdfd78af_1 bash

✅ Via CLI

The files dumped in multiqc_data have the yaml extension as expected.

$ multiqc -f -k yaml .

  /// MultiQC 🔍 | v1.13.dev0

|           multiqc | Search path : /home/harshil/test
|         searching | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1  
|            fastqc | Found 1 reports
|           multiqc | Compressing plot data
|           multiqc | Report      : multiqc_report.html
|           multiqc | Data        : multiqc_data
|           multiqc | MultiQC complete

$ ls multiqc_data/
multiqc.log  multiqc_citations.yaml  multiqc_data.json  multiqc_fastqc.yaml  multiqc_general_stats.yaml  multiqc_sources.yaml

❌ Via Config

The files dumped in multiqc_data have the txt extension indicating that there may be an issue resolving the configuration.

$ rm -r multiqc_*

$ echo 'data_format: "yaml"' > multiqc_config.yaml

$ multiqc -f .

  /// MultiQC 🔍 | v1.13.dev0

|           multiqc | Search path : /home/harshil/test
|         searching | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 2/2  
|            fastqc | Found 1 reports
|           multiqc | Compressing plot data
|           multiqc | Report      : multiqc_report.html
|           multiqc | Data        : multiqc_data
|           multiqc | MultiQC complete

$ ls multiqc_data/
multiqc.log  multiqc_citations.txt  multiqc_data.json  multiqc_fastqc.txt  multiqc_general_stats.txt  multiqc_sources.txt

File that triggers the error

No response

MultiQC Error log

No response

[ewels]

So it looks like I broke this in fc65e7f when I was messing around trying to get the CLI help to look nicer. I think I added the default there so that it would show. However, that's before the config is fully loaded, so it always loads it as TSV whatever you have in other config files. Then, Click uses that default to pass it onwards and MultiQC thinks that you've set it manually on the CLI so it overwrites the actually fully parsed config. It still works with -k because that takes precedence over the Click default. Phew 😅

Easy fix, just remove the Click default.

[ewels]

Fixed in abf02c7

[drpatelh]

Nice! Thanks ❤️