Hands-on tutorial on Nextflow and Containers (Docker and Singularity). Paris 2018. For this tutorial we need to install Nextflow (https://www.nextflow.io/), Singularity (http://singularity.lbl.gov/) and Docker (https://www.docker.com/).
It is a Debian Stretch machine from OSBoxes project (ref: https://www.osboxes.org/debian/). Java, Docker, Singularity and Nextflow are already installed.
- OVA: http://biocore.crg.eu/courses/C4LWG-2018/C4LWG-2018-full.ova
- OVA md5: http://biocore.crg.eu/courses/C4LWG-2018/C4LWG-2018-full.ova.md5
Check appliances download
Take care that files downloaded correctly (around 6GB). You can check with MD5 utilites from the terminal
$ md5sum -c C4LWG-2018-full.ova.md5 C4LWG-2018-full.ova: OK
$ md5 C4LWG-2018-full.ova $ cat C4LWG-2018-full.ova.md5
Check both outputs show the same string.
- User: osboxes
- Password: osboxes.org
If you need to use the root user (e.g. via
su -l), it has the same password.
- For installing NextFlow we need Java version 1.8. You can check with "java -version". Then just type "curl -s https://get.nextflow.io | bash" for installing a local copy in your current directory. Finally type "./nextflow run hello" for testing.
- Mac OS X users can consider installing Homebrew and Homebrew-Cask.
- Docker (Community Edition): https://www.docker.com/community-edition . Download and install last stable version in your system.
- Cask users:
brew cask install docker
- Cask users:
curl -s https://get.nextflow.io | bash
You can retrieve Docker image with all used software by doing:
docker pull biocorecrg/c4lwg-2018
Alternately, you can always modify and build a Docker image yourself in your computer by doing:
docker build -t myimagename .
Converting Docker image into Singularity image
singularity build c4lwg-2018.simg docker://biocorecrg/c4lwg-2018
- If you experience problems executing the generated image, e.g.,
ERROR : No valid /bin/sh in container, try to change your umask (e. g.,
umask 000) Ref
Generating a Singularity image from a Singularity recipe
singularity build c4lwg-2018.xenial.simg Singularity.xenial
We can reach the first folder test0
cd test0; ls
We can have a look at the code and launch it:
nextflow run test0.nf
Nextflow creates a directory named work with different subfolders. Each one contains the input, output and some hidden files:
In this case there is neither input nor output file.
Second example where we read a fasta file, split it in several ones and tests on them.
cd test1; ls
nextflow run test1.nf In the work folder we have subfolders containing this time a link to the input and the output file. In output folder we have links to the final results.
Third example where we launch two fastQC analysis and we run multiQC on their result: cd test2; ls
- params.config: with parameters
- nextflow.config: with information about resources needed for each task and the container to be used
We can inspect the different files and launch te pipeline.
nextflow run test2.nf -bg We can inspect the results in the different folders.