Operations and Troubleshooting

Operations and troubleshooting

This page covers resource configuration, retries, degraded caller mode and the errors most often seen before or during a run. Parameter defaults are in Configuration.

Resources

Every process uses one of three labels:

Label	Typical processes
process_low	index creation, decompression, coverage, mapping statistics
process_medium	mapping, calling, scattering, consensus, merge, export
process_high	GATK HaplotypeCaller

resources.low, resources.medium and resources.high set cpus, memory_mb and time. resources.limits caps the values after retry escalation. mapping.cpu, calling.cpu and consensus.cpu remain authoritative for their stages and override the label CPU defaults.

Memory and time are multiplied by the task attempt number. A task that exits with status 137 (out-of-memory) or 140 (walltime) is retried up to resources.max_retries times with the escalated request. JVM heaps are derived from the resulting task memory, including GATK processes started by GNU parallel.

Mapping memory

The mapper and BAM post-processing run concurrently inside one task. mapping.cpu is split between them, and the post-processing share is split again across its samtools stages.

samtools sort runs while the mapper index is resident. Sort receives half of the task memory that remains after mapping.aligner_reserve_mb. When the reserve is 0, it is estimated from the staged index files plus 1 GiB. For large genomes set it explicitly (roughly 4 GiB for a human Bowtie2 index, 6 GiB for BWA and 8 GiB or more for a Minimap2 .mmi) and raise resources.medium.memory_mb so the reserve, sort buffer and streaming stages fit together. A reserve that is not smaller than the task memory stops the task with a message naming both values.

Caller failures and dropped samples

• Transient caller failures (137/140) are retried up to resources.max_retries.
• Other caller failures terminate the run by default.
• calling.allow_caller_failure: true continues with surviving callers. The run still aborts if consensus.threshold exceeds the number of survivors.
• A sample whose every caller fails is dropped with a warning.
• coverage.on_empty: "fail" aborts on an empty coverage BED; "skip" drops the sample with a warning.
• Dropped samples are published in dropped_samples.txt and, for merged output, in SamplesDropped in the ##ChoCallate= header.

See Consensus model for the voting behavior in degraded mode.

Merge file descriptors

For output.type: "single" the pipeline checks the soft open-file limit before the first task. The required descriptor count is computed from the sample count and consensus.cpu. If the limit is too low, the run aborts with the required value:

ulimit -n
ulimit -n 4096

See Merge strategies and testing for the formula and examples.

Troubleshooting

No ... index files found ... missing: ...

The mapper index is incomplete or the prefix is wrong. Check input.reference_index_dir and input.reference_index_prefix. The default prefix is the reference file name including .gz. Bowtie2 requires a complete .bt2 or .bt2l set; BWA requires all five index files; Minimap2 requires the .mmi.

reference is gzip-compressed but not BGZF

Recompress the FASTA with bgzip, or provide a plain FASTA. Plain gzip is not supported by samtools faidx or GATK.

BAM header @SQ entries do not match reference / M5 checksums do not match

The BAM was aligned to a different reference. Provide the reference used for alignment, or re-align the reads. When the BAM has no M5 entries, names and lengths are still checked.

Sample dropped: coverage.bed is empty

No base reaches coverage.min_coverage. Lower the threshold, widen the include BED or check the BAM/reference pair. Set coverage.on_empty: "skip" to drop such samples instead of aborting the run.

All callers failed for a sample

Inspect the caller task logs and extra_args. With calling.allow_caller_failure: false (default), a deterministic caller error terminates the run. With true, the sample is dropped when no caller survives, and the run still enforces consensus.threshold against the survivors.

Too many open files during merge

Raise ulimit -n, or lower consensus.cpu so fewer merge chunks and FIFOs are opened. The preflight check reports the required value.

WARN: Unrecognized config option 'boost.cleanup'

This warning can be printed by the Nextflow config validator even though nf-boost applies boost.cleanup. The cleanup behavior is covered by tests/python/test_boost_cleanup.py. Set boost.cleanup: false in the params file when you need -resume to reuse completed tasks.

Broken links after output.publish_mode: "link"

Linked results point into the work directory. Retain the work directory (boost.cleanup: false) or use the default copy mode.

Offline run cannot resolve plugins

For a Conda installation, install the pinned nf-schema@3.0.0 and nf-boost@0.8.0 plugins once on a connected machine. The Docker image pins both plugins at build time and supports --network none.

Logs

• .nextflow.log in the launch directory: run-level events and process failures.
• trace.txt: per-process status and resource usage.
• pipeline_report.html and timeline_report.html: execution and timing views. The reports follow the paths configured in nextflow.config; see Output structure for the params-file caveat.
• Nextflow work/ directories: task inputs, commands and stderr. Failed tasks keep their staged inputs and temporary files for debugging.