Skip to content
Calib clusters barcode tagged paired-end reads based on their barcode and sequence similarity.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
aux Moved bam-readcount to aux/othertools; git ignoring SPOA and CMake li… Sep 12, 2018
clustering Added -g to help Jun 28, 2019
consensus Bugfix: was adding qual chars for gaps Nov 28, 2018
experiments Typo in default parameter setting Jun 25, 2019
simulating Generating from all over the genome if no panel is given Jun 28, 2019
slurm_scripts Update README.md Jun 28, 2019
.gitignore
.gitmodules
LICENSE Comments, comments all over the place Dec 13, 2017
Makefile Make Makefile better Jun 28, 2019
README.md Update README.md Jan 21, 2019
benchmark Bugfix in real slurm tests script; umitools not umi-tools Sep 6, 2018
calib.cc Ignoring debug files Jun 28, 2019
environment.yml Add conda env YAML file Jun 28, 2019

README.md

Calib

Calib clusters paired-end reads using their barcodes and sequences. Calib is suitable for amplicon sequencing where a molecule is tagged, then PCR amplified with high depth, also known as Unique Molecule Identifier (UMI) sequencing.

Calib stands for Clustering without alignment using (locality sensitive hashing) LSH and MinHashing of barcoded reads. Calib comes for the Arabic word قالب /IPA:qaːlib/ which means template and is a reference to Calib's use of LSH templates.

Prerequisites

Calib clustering

Calib main module has one prerequisite:

  • GCC with version 5.2 or higher

Calib error correction

Calib error correction depends SPOA v1.1.3 which in turn depends on CMake v3.2 or higher. However, Calib error correction installation script automatically detects if cmake in the $PATH is CMake v3.2 or higher. If it's not, then it downloads CMake and v3.12, installs it. Then the installation script will clone SPOA v1.1.3 and install it using CMake.

Calib Simulation

Calib simulation module has some Python3 prerequisites that can be easily satisfied using Conda package manager:

Finally, if you want to generate the different plots (check this README) you need to also have:

Which can be also easily installed using Conda.

Installing Calib

First, clone this repository:

git clone https://github.com/vpc-ccg/calib.git calib
git checkout v0.2

To install Calib clustering module:

cd calib
make
cd ..

To install Calib error correction module:

cd calib
make -C consensus/
cd ..

Running Calib

Clustering

To run Calib clustering, run:

cd <CALIB_DIRECTORY>
./calib -f <reads_1> -r <reads_2> -l <barcode_tag_length> -o <output_file_prefix>

For example:

cd calib
./calib -f R1.fastq -r R2.fastq -l 8 -o R.

Calib will cluster the reads in <reads_1> and <reads_2> FASTQ files that are tagged with barcode tags of length <barcode_tag_length>. Note that this tag length of the length of barcode tag on one mate of the paired-end reads. The output filename will be <output_file_prefix>cluster.

Output format

The output file will contain one line per input read. Each record is tab separated with the following columns:

  1. read_cluster_id: Consecutive integers starting at 0 and ending at number of clusters - 1
  2. read_node_id: Consecutive integers starting at 0 and ending at number of nodes - 1
  3. read_id: 0-based order of the read in the input files
  4. read_f_name: FASTQ name of the read's forward mate
  5. read_f_seq: FASTQ sequence of the read's forward mate
  6. read_f_qual: FASTQ quality sequence of the read's forward mate
  7. read_r_name: FASTQ name of the read's reverse mate
  8. read_r_seq: FASTQ sequence of the read's reverse mate
  9. read_r_qual: FASTQ quality sequence of the read's reverse mate

Clustering parameters

Calib clustering has different clustering parameters that can be changed manually from the default pre-configuration:

  • --error_tolerance or -e: positive integer no larger than l, the barcode tag length
  • --kmer-size or k: positive integer
  • --minimizer-count or -m: positive integers
  • --minimizer-threshold or t: nonnegative integer no larger than m

Changing these parameters is might not be very obvious. We recommend checking with our parameter selection experiments before doing so.

Clustering multithreading

Calib clustering is can run multi-threaded using:

  • --threads or c: positive integer no larger than 8

Please check our thread scalability experiments to have an idea on the time vs. memory tradeoff of Calib clustering multithreading.

Other clustering parameters

Finally, Calib clustering has these parameters that are added for convenience:

  • --ignored-sequence-prefix-length or p: nonnegative integer for the number of bases to ignore in clustering after the barcode tag in the read sequences.
  • --no-sort: A flag to tell Calib to keep the original order of the reads in the output file instead of grouping the reads of the same clusters together.

Error Correction

To run Calib error correction, run:

cd <CALIB_DIRECTORY>
./consensus/calib_cons -c <cluster_file> -q <space_separated_FASTQ_list> -o <space_separated_output_prefix_list>

For example:

cd calib
./consensus/calib_cons \
	-c R.cluster \
	-q R1.fastq R2.fastq \
	-o R1. R2.

Output format

Calib error correction will output two files per input FASTQ file. One file will be a FASTQ file containing one record per consensus generated. The second file will contain multiple sequence alignment (MSA) of the cluster sequences.

Error correction parameters

Error correction has one parameter:

  • --min-reads-per-cluster or -m: positive integer for the minimum number of reads required in a cluster to output the cluster consensus. Default is 2.
  • --threads or t: positive integer for number of threads to use. Default is 4.

Simulation module

Calib has a simulation molecule that generates paired-end UMI tagged reads. The simulation pipeline is Calib's Makefile itself. It generates the following components:

  • panel: A BED file containing the exons coordinates of a list of genes. Its Make variables are:
    • annotation: GTF annotation file
    • gene_list: Text file containing set of gene names from annotation, one per line.
    • num_genes: Number of genes to sample from gene_list to be selected for making panel.
  • molecules: A FASTA file containing randomly generated molecules that overlap with the regions in panel. Its Make variables are:
    • molecule_size_mu: Average size of generated molecule
    • molecule_size_dev: Standard deviation of the size of the generated molecule.
    • min_molecule_size: Minimum size cutoff for dropping any generated molecule
    • num_molecules: Number of molecules to generate, after any dropouts due to min_molecule_size
  • barcodes: Text file containing a set of barcode tags of the same length, one per line. Its Make variables are:
    • num_barcodes
    • barcode_length
  • barcoded_molecules: A FASTA file with molecules randomly tagged with random barcode tag from barcodes, one barcode tag for either end.
  • amplified_barcoded_molecules: A FASTA file containing PCR amplified barcoded_molecules. Its Make parameters are:
    • pcr_cycles: Number of PCR cycles to perform
    • pcr_duplication_rate: Percentage of molecules to be selected for duplication in each PCR cycle from last PCR cycle.
    • pcr_error_rate: PCR substitution error rate per duplicated base in each PCR cycle.
  • simulate: A Make target to generate paired-end read FASTQ files. It has the following Make variables:
    • sequencing_machine: ART Illumina sequencing machine.
    • read_length: Read mate length to be generated

Since Calib simulation pipeline is basically a Makefile, any target that depends on the previous targets inherits its variables. For example:

make simulate num_molecules=1000

Will generate paired-end reads using all the default simulation parameters (check Makefile header) but with num_molecules of 1000.

Citation

Baraa Orabi, Emre Erhan, Brian McConeghy, Stanislav V Volik, Stephane Le Bihan, Robert Bell, Colin C Collins, Cedric Chauve, Faraz Hach; Alignment-free clustering of UMI tagged DNA molecules, Bioinformatics, , bty888, https://doi.org/10.1093/bioinformatics/bty888

Reporting issues and bugs

If you have any issues, questions, or bug reports, please open an issue and will try to address promptly.

You can’t perform that action at this time.