Use -params-file params.yml to pass parameters instead of -c custom.config

[poojasgupta]

Description of the bug

I am getting an error when running the latest version of viralrecon (v2.3.1). I re-ran a subset of my samples with the new version of viralrecon and got this error. The pipeline completes fastp, fastqc steps but throws this error at the bowtie alignment step. These samples ran successfully with the previous version of viralrecon.

Command used and terminal output

Command:
nextflow run nf-core/viralrecon --input ${run_name}_samplesheet.csv -c test_viralrecon.config \
                                -profile singularity \
                                --outdir $out_dir1 \
			        --multiqc_title ${run_name}_viralrecon

Output:
Error executing process > 'NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:SAMTOOLS_SORT (220124_SV3_UT_A01290_220208_S585_L002)'

Caused by:
  Input and output names are the same, use "task.ext.prefix" to disambiguate!

Source block:
  def args = task.ext.args ?: ''
  def prefix = task.ext.prefix ?: "${meta.id}"
  if ("$bam" == "${prefix}.bam") error "Input and output names are the same, use \"task.ext.prefix\" to disambiguate!"
  """
      samtools sort $args -@ $task.cpus -o ${prefix}.bam -T $prefix $bam
      cat <<-END_VERSIONS > versions.yml
      "${task.process}":
          samtools: \$(echo \$(samtools --version 2>&1) | sed 's/^.*samtools //; s/Using.*\$//')
      END_VERSIONS
      """

Relevant files

test_viralrecon_config.txt

System information

Nextflow version (21.10.6)
Container engine: Singularity
Version of nf-core/viralrecon (2.3.1)

[afonsoguerra]

Just to add from my slack comment. Here are my two command lines I use. The first one fails, the second one completes without error. The parameters specified in the command line in the second scenario are passed (and apparently recognised) in the first scenario via a nextflow.config file

nextflow run viralrecon/main.nf -profile singularity -with-tower --input ${run}_sampleSheet.csv --gff refannot.gff --fasta refseq.fa --outdir ${run}_Results/

nextflow run viralrecon/main.nf -resume -profile singularity --skip_kraken2 --skip_assembly --input ${run}_sampleSheet.csv -with-tower --platform illumina --protocol 'amplicon' --genome 'MN908947.3' --primer_set 'artic' --primer_set_version 4 --filter_duplicates --variant_caller ivar --consensus_caller ivar --gff refannot.gff --fasta refseq.fa --outdir ${run}_Results/

Nextflow: 21.10.6
nf-core/viralrecon: '2.3'

[drpatelh]

Hi @poojasgupta @afonsoguerra ! Thanks for reporting.

These errors are related to the way that we have re-structured the module configuration in the latest Nextflow DSL2 syntax we have adopted on nf-core and won't be specific to this pipeline:

• We can no longer use a custom config provided via the Nextflow -c to overwrite params values in the pipeline. This is because the custom config provided via -c will be loaded AFTER the modules.config containing all of the module level configuration is loaded and hence the parameters won't be passed to the pipeline and it will fail. See Nextflow configuration docs for configuration priority order.
• As natively intended, the -c parameter must only be used to provide process or other configuration related settings to the pipeline.
• We need to now use either a -params-file params.yml / CLI to pass params to the pipeline as shown below:

Command

nextflow run nf-core/viralrecon \
    --input samplesheet.csv \
    --outdir ./results \
    -params-file params.yml \
    -profile docker \
    -r 2.3.1

params.yml

platform: 'illumina'
protocol: 'amplicon'
primer_set: 'artic'
primer_set_version: 4
genome: 'MN908947.3'
kraken2_db: '/home/harshil/kraken2_human.tar.gz'
kraken2_variants_host_filter: true
min_mapped_reads: 500
save_mpileup: true
variant_caller: 'ivar' 
consensus_caller: 'ivar'
skip_assembly: true
max_memory: '6.GB'

I stripped out the params you were using before @poojasgupta and providing them via params.yml works for me.

We still need to add some proper documentation for this which we will prioritise during the upcoming Hackathon in March.

[mahesh-panchal]

I made an issue on Nextflow as it's happening very often:
nextflow-io/nextflow#2662

[drpatelh]

@poojasgupta confirmed that this approach worked for her so will close this for now. I have pinned it as an issue for more general reference.