We have a new bioinformatic resource that largely replaces the functionality of this project! See our new repository here: https://github.com/epi2me-labs/wf-isoforms

This repository is now unsupported and we do not recommend its use. Please contact Oxford Nanopore: support@nanoporetech.com for help with your application if it is not possible to upgrade to our new resources, or we are missing key features.

Pipeline for de novo clustering of long transcriptomic reads

The first natural step in the de novo analysis of long transcriptomic data in the absence of a reference genome is the clustering of the reads into groups corresponding to gene families. This pipeline performs that task using the isONclust2 tool, which is based on the approach pioneered by isONclust using minimizers and occasional pairwise alignment.

Since isONclust2 is implemented in C++ using efficient data structures, it can distribute computing across multiple cores and machines, thus it is able to cope with large transcriptomic datasets generated using PromethION P24 and P48 flow cells.

The pipeline optionally concatenates FASTQ files in the MinKNOW/guppy output and performs the optional trimming and orientation of cDNA reads using pychopper.

The main output of the pipeline is the assignment of read identifiers to gene clusters and the clustered reads grouped into one FASTQ file per gene cluster. This output is fit for downstream, gene level analysis.

Getting Started

Input

• The input is a file with fastq records or a directory of containing fastq files, which is specified is config.yml.

Output

The main output files generated by the pipeline are under output_directory/final_clusters:

• batch_info.tsv - information on the final output batch
• cluster_cons.fq - a fastq file with the cluster representatives (or consensuses)
• cluster_fastq/ - a directory of fastq files (one per gene cluster)
• clusters_info.tsv - A TSV file with the cluster sizes
• clusters.tsv - A TSV file with read identifiers assigned to clusters

Dependencies

• miniconda
• The rest of the dependencies are installed via conda.
• CPU with AVX/SSE extensions; the workflow has been validated using the GridION device.

Installation

Clone the pipeline and the pipeline toolset by issuing:

git clone --recursive https://github.com/nanoporetech/pipeline-nanopore-denovo-isoforms.git

Install the dependencies using conda into a new environment:

conda env create -f env.yml

Activate the conda environment:

conda activate denovo-isoforms

Usage

Edit config.yml to set the input fastq and parameters, then on a local machine issue:

snakemake -j <num_cores> all

For analysing larger datsets (e.g. a PromethION flow cell) it is advisable to run the pipeline on a SGE cluster through DRMAA:

snakemake --rerun-incomplete -j 1000 --latency-wait 600 --drmaa-log-dir sge_logs --drmaa ' -P project_name -V -cwd -l h_vmem=200G,mem_free=155G -pe mt 5' all

Results

The evaluation metrics (also described in the isONclust paper - free version) reported are:

• Homogeneity
• Completeness
• V-measure
• Adjusted Rand Index - ARI

The clustering has a binary classification issue (i.e. a single read is either correctly or incorrectly "labelled" by the algorithm given a ground truth). Each read must instead be evaluated in relation to the reads in the same and other clusters (e.g. which pairs or reads are "correctly assigned to the same cluster?" and "erroneously assigned to different clusters?"). From this, common measures such as precision, recall, and F-score cannot be used. The Homogeneity, completeness, and V-measure are analogous to the precision, recall, and F-score measures for binary classification issues, but are adapted for clustering issues.

Intuitively, homogeneity (i.e. precision) penalizes over-clustering, i.e. wrongly clustering together reads, while completeness (i.e. sensitivity) penalizes under-clustering, i.e. mistakenly keeping reads in different clusters. The V-measure is then defined as the mean of homogeneity and completeness (i.e. F-measure). We also include the commonly used Adjusted Rand Index (ARI). Intuitively, ARI measures the percentage of read pairs correctly clustered, normalized so that a perfect clustering achieves an ARI of 1 and a random cluster assignment achieves an ARI of 0. Briefly, both of these clustering quality metrics are derived from computing pairwise correct and incorrect groupings of reads, instead of individually classifying a read as correct or incorrect (as in classification issues).

Performance on PCS109 SIRV data

The performance on ~19k SIRV E0 reads generated using the PCS109 protocol can be assesed by running the evaluation script:

./run_evaluation.sh

The main results are:

Performance on PCS109 Drosophila melanogaster data

The performance on a D. melanogaster datasets generated using the SQK-PCS109 protocol can be assesed by running the evaluation script:

./run_evaluation_dmel.sh

The main results are:

Acknowledgements

This software was built in collaboration with Kristoffer Sahlin and Paul Medvedev.

Licence and Copyright

(c) 2020 Oxford Nanopore Technologies Ltd.

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

FAQs and tips

References and Supporting Information

See the post announcing transcriptomics tools in the Nanopore Community here.

Research Release

Research releases are provided as technology demonstrators to provide early access to features or stimulate Community development of tools. Support for this software will be minimal and is only provided directly by the developers. Feature requests, improvements, and discussions are welcome and can be implemented by forking and pull requests. However much as we would like to rectify every issue and piece of feedback users may have, the developers may have limited resource for support of this software. Research releases may be unstable and subject to rapid iteration by Oxford Nanopore Technologies.