How to use Singularity

This tutorial explains how to use YAMP with Singularity.

YAMP and Singularity

YAMP takes advantage of a multi-image scenario. This means that each process will specify which container should be used, along with its version (as explained in the Multi image scenario tutorial).

YAMP also provides a singularity profile that can be used to tell Nextflow to enable the use of Singularity, as in the following example:

nextflow run YAMP.nf --reads1 myfile_R1.fq.gz --reads2 myfile_R2.fq.gz --prefix my_sample  
   --outdir output_folder --mode complete -profile base,singularity

The singularity profile is defined (along with all the other Nextflow YAMP profiles) in the nexflow.config file as follows:

singularity.enabled = true

Please note that, by default, Nextflow caches Singularity images in the singularity directory (located within the pipeline work directory). Nextflow developers suggest to provide a centralised caching directory by using either the NXF_SINGULARITY_CACHEDIR environment variable or the singularity.cacheDir setting in the nextflow.config file.

In you want to take advantage of the environment variable, you have to set it in the command line (remember to export it!) before running YAMP, as follows:

export NXF_SINGULARITY_CACHEDIR=/path/to/folder/with/singularity/images

nextflow run YAMP.nf --reads1 myfile_R1.fq.gz --reads2 myfile_R2.fq.gz --prefix my_sample  
   --outdir output_folder --mode complete -profile base,singularity

If you want to take advantage of the singularity.cacheDir setting, you should modify the nextflow.config file (singularity profile) as follows:

singularity {
	singularity.enabled = true
	singularity.cacheDir = /path/to/folder/with/singularity/images
}

Please also remember that, when using a computing cluster the Singularity cache directory must be accessible to all computing nodes.

A word of caution

Nextflow should be installed independently as explained here.

Nextflow orchestrates, in a transparent fashion, the flow of the pipeline by wrapping and executing each step using the singularity run command. Thus, Nextflow lies outside the container, which is responsible for instantiating.

You can find more information about Singularity containers and Nextflow here.

Retro compatibility

• YAMP multi-image scenario and singularity profile were introduced in version 0.9.5. The yampdocker file provided with previous versions is no longer compatible but mostly outdated and should not be used. If this is a problem for you, please open an issue and we will try to help.