Introduction

rewarewaannotation is a bioinformatics pipeline built by Ann McArtney and ported into Nextflow by Katie Herron, originally developed for the annotation of the rewarewa (Knightia excelsa) genome. The pipeline takes paired-end RNA-seq reads and a genome assembly as input and conducts QC, trimming and alignment. The target genome is repeat masked prior to both the genome and RNA-seq evidence being given as input to BRAKER3.

Default steps in the pipeline:

1. Merge re-sequenced FastQ files (cat)
2. Read QC (FastQC)
3. Adapter and quality trimming (Trim Galore!)
4. Trimmed read QC (FastQC)
5. OPTIONAL:
   • a. Hardtrimming (Trim Galore!)
   • b. Hardtrimmed read QC (FastQC)
6. Alignment (STAR)
7. Alignment summary metrics (picard CollectAlignmentSummaryMetrics)
8. Assembly QC (BUSCO)
9. Build custom repeat database (RepeatModeler)
10. Mask repeats in genome assembly (RepeatMasker)
11. Genome annotation (BRAKER3)
12. Annotation QC (BUSCO)
13. Annotation summary metrics (agat spstatistics)
14. Present QC for raw reads, alignment and annotation (MultiQC)

Usage

Pipeline usage is covered more comprehensively on this page.

Preparing to run on NeSI

1. Make a new directory for your annotation workflow. git clone this repo. Then make a directory for the annotation output.

2. Install NextFlow locally on NeSI as per the Introduction to Nextflow workshop (see 'How to install Nextflow locally'). You may need to load Java:

   module load Java/11.0.4

3. Once you have moved NextFlow to your $HOME/bin, check whether it can be found:

   nextflow -version

   If it doesn't return NextFlow version information, you will need to export bin to path:

   export PATH="$HOME/bin:$PATH"

   It's probably a good idea to add this to your ~/.bashrc.

4. Set up the rest of the environment ready to run the test config. We are running this pipeline with Singularity - please ignore the message regarding it being deprecated on NeSI.

   We will need to set up cache and temporary directories (e.g., /nesi/nobackup/landcare03691/singularity-cache, /nesi/nobackup/landcare03691/tmp-anno, and run setfacl -b commands to bypass NeSI security access control on nobackup to allow pull from online repos.

   For repeat usage, we recommend adding the following to your ~/.bashrc:

## NextFlow set up for annotation pipeline
export PATH="${HOME}/bin:$PATH"
export NXF_TEMP=/path/to/tmp-anno
export NXF_HOME=~/.nextflow
export NXF_SINGULARITY_CACHEDIR=/path/to/singularity-cache
export SINGULARITY_CACHEDIR=/path/to/singularity-cache
export SINGULARITY_TMPDIR=/path/to/tmp-anno

Then prior to running the pipeline, you only need to do the following:

module load Java/11.0.4
nextflow -version
module load Singularity/3.11.3
setfacl -b "${NXF_SINGULARITY_CACHEDIR}" /path/to/rewarewaannotation/main.nf
setfacl -b "${SINGULARITY_TMPDIR}" /path/to/rewarewaannotation/main.nf

8. Test your setup.

   Make sure to test your setup with -profile test before running the workflow on actual data. Note: The nextflow process will also make a .nextflow.log file which is more detailed than your anno-test.log, but if you don't rename it, it will be overwritten.

   nextflow run /path/to/rewarewaannotation/ -profile test,singularity --outdir results &> anno-test.log

The test should complete, logging one error, which will be a result of the test dataset being too small to run Braker3 - this is an expected error! If there are no other errors or warnings, you should be good to go.

You can test resuming the pipeline using -resume. Once you have run the test, I recommend cleaning out the singularity cache prior to starting a run with real data.

singularity cache clean

Setting up to run for your data on NeSI via SLURM

In brief:

Prepare a samplesheet with your RNA-seq input data that looks as follows:

samplesheet.csv:

sample_id,file1,file2
SAMPLE_1,SAMPLE_1_R1.fastq.gz,SAMPLE_1_R2.fastq.gz,
SAMPLE_2,SAMPLE_2_R1.fastq.gz,SAMPLE_2_R2.fastq.gz,
<...>

Prepare a params.yml file for your input data: rewarewa_params.yml

input                         : 'samplesheet.csv'
outdir                        : 'results'
assembly                      : '<ASSEMBLY_FASTA_PATH>'
assembly_name                 : 'asmName'
busco_lineages                : 'eukaryota_odb10,embryophyta_odb10'

Each row represents a pair of fastq files (paired end). This pipeline only accepts paired-end reads. Input files can be compressed or uncompressed. Re-sequenced samples will be merged into a single fastq file at the start of the pipeline.

Now, you can run the pipeline on the command line using:

nextflow run kherronism/rewarewaannotation \
   -profile <docker/singularity/.../institute> \
   --input samplesheet.csv \
   --assembly <ASSEMBLY_FILE> \
   --assembly_name <ASSEMBLY_NAME> \
   --outdir <OUTDIR>

However, we want to make use of the SLURM scheduler and allow NextFlow to make its own arrays to handle the various pipeline steps.

Running on SLURM

Along with making your samplesheet, you will need a SLURM script like this example: NeSI_slurm.sh. We have modified the sonic.config to run on NeSI.

For a full breakdown of available params for the pipeline see this page.

To re-run the pipeline with the parameters used for rewarewa (example samplesheet.csv and params file given in test-datasets/kniExce folder):

  nextflow run kherronism/rewarewaannotation \
   -profile <docker/singularity/.../institute> \
   -params-file test-datasets/kniExec/rewarewa_params.config

Also included in test-datasets/kniExce are a few helper files:

• environment.yml: For creating a conda environment for Nextflow and nf-core.
• rewarewa_params.yml: A params file containing the parameters for reproducing the rewarewa annotation.
• rewarewa_slurm.sh: Example of how to submit the pipeline run as a job on a slurm cluster. However, this does require giving Nextflow an additional config file, which is dependent on the set-up of the institutional cluster. See this page and nf-core/configs for more details.
  • As an example, the sonic.config file in test-datasets/kniExce is a config file for the Sonic HPC cluster.

Warning: Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.

For a breakdown of the outputs of the pipeline see this page.

Credits

Pipeline originally written and implemented by Ann McCartney and ported to Nextflow by Katie Herron.

Modifications for use on NeSI by Nat Forsdick, Dinindu Senanayake, and Chris Hakkaart.

Citations

An extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.

This pipeline uses code and infrastructure developed and maintained by the nf-core community, reused here under the MIT license.

The nf-core framework for community-curated bioinformatics pipelines.

Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.

Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.