Running PhyloPHoeNIx

Input Parameters

The following are the possible parameters you can pass to PhyloPHoeNIx. You can get this screen by running:

nextflow run cdcgov/phylophoenix -r v1.0.0 --help

Input Samplesheet

You will need to create a samplesheet with information about the samples you would like to analyze before running the pipeline. Use the --input parameter to specify its location. It must be a comma-separated file (csv) with at least 2 columns and a header row, as shown in the example below. DO NOT HAVE ANY SPACES IN THIS FILE. Do make sure the paths are full paths and not relative. For best results, use the automated samplesheet creation scripts described in the PHoeNIx Wiki.

--input '[path to samplesheet file]'

The samplesheet can have as many columns as you desire; however, there is a strict requirement for the first 2 columns to match those defined in the table below. When PHoeNIx runs, it creates a file Directory_samplesheet.csv for all the isolates in that project folder. This file can be used to easily copy and paste portions into a new samplesheet to run samples from different PHoeNIx project folders.

A final samplesheet file should look something like the one below.

sample,directory
SAMPLE_1,$PATH_TO_PHX_PROJECT/SAMPLE1_FOLDER
SAMPLE_2,$PATH_TO_PHX_PROJECT/SAMPLE2_FOLDER
SAMPLE_3,$PATH_TO_PHX_PROJECT/SAMPLE3_FOLDER

Column	Description
sample	Custom sample name. Spaces in sample names are automatically converted to underscores (_).
directory	Full path to a directory where sample folder in a PHoeNIx output folder is found.

An example samplesheet has been provided with the pipeline and can be used for testing.

Metadata Sheet

Currently, PhyloPHoeNIx is set up so its output can be easily imported into Microreact, however, the tree file can also be uploaded into other platforms like iTol. Currently, the metadata file (TSV file) produced from PhyloPHoeNIx is not directly compatible with iTol and will be available in future versions. If you want to include metadata for visualization in Microreact or another program, use the --metadata parameter to specify its location like:

--metadata '[path to metadata file]'

The metadata file should be TSV formatted.

Column	Description
sample	Custom sample name. ALL samples should be accounted for in the sheet. If some samples have no metadata, just have a row for that sample and all the columns blank.
date	Date that will be used for creating a timeline in Microreact. Format should be mm/dd/yyyy and this will be converted to a form that Microreact likes better (optional).
country	Country of sample origin, this will be used to get latitude and longitude for mapping in Microreact (optional).
state	State of sample origin, this will be used to get latitude and longitude for mapping in Microreact (optional).
county	County of sample origin, this will be used to get latitude and longitude for mapping in Microreact (optional).
city	City of sample origin, this will be used to get latitude and longitude for mapping in Microreact (optional).
anything_else	You can have any number of columns named whatever you want. These column values can be used to color isolates during visualization so details like source, facility, room and any other epidemiological data can be helpful in understanding how your samples relate to one another.

If you have city or county in your metadata file, you will be required to provide country and state information just to ensure the mapping is accurate. Admittedly, this code is currently biased for locations in the USA, as most of the testing focused on making sure those locations were mapped correctly. Future versions will have added support for a "regions" column for international locations with regions rather than "states".

Outputs

Output file structure

The project-level output (phylophoenix_output is the project folder) of PhyloPHoeNIx is structured like the following:

📦phylophoenix_output
┣ 📂<ST>
┃ ┣ 📂<sample_id>
┃ ┃ ┣ 📜<sample_id>.bam
┃ ┃ ┣ 📜<sample_id>_sorted.bam
┃ ┃ ┣ 📜<sample_id>_filtered_density.txt
┃ ┃ ┣ 📜<sample_id>_freebayes_filtered.vcf.gz
┃ ┃ ┣ 📜<sample_id>_freebayes.vcf
┃ ┃ ┗ 📜<sample_id>_mpileup.vcf.gz
┃ ┣ 📜<ST>_centroid_info.txt
┃ ┣ 📜<ST>_cleaned_metadata.tsv --> upload to Microreact
┃ ┣ 📜<ST>_phylogeneticTree.newick --> upload to Microreact/iTol or another visualization program
┃ ┣ 📜<ST>_snvAlignment.phy
┃ ┣ 📜<ST>_snvMatrix.tsv --> upload to Microreact
┃ ┣ 📜<ST>_vcf2core.tsv
┃ ┣ 📜filtered_density_all.txt
┃ ┣ 📜filterStats.txt
┃ ┣ 📜mappingQuality.txt
┃ ┣ 📜new_invalid_positions.bed
┃ ┗ 📜phylogeneticTreeStats.txt
┣ 📂ST_SampleSheets
┃ ┗ <ST>_samplesheet.csv
┣ 📂pipeline_info
┃ ┣ 📜execution_report_.html
┃ ┣ 📜execution_timeline_.html
┃ ┣ 📜execution_trace_.txt
┃ ┣ 📜pipeline_dag_.html
┃ ┣ 📜samplesheet.valid.csv
┃ ┗ 📜software_versions.yml
┣ 📜Directory_samplesheet.csv
┣ 📜SNVPhyl_GRiPHin_Summary.xlsx
┗ 📜GRiPHin_Summary.xlsx

Output File Overview

The following is an explanation of the files that are output:

• ST - this contains all the files produced for this sequence type, some of which come from output produced from SNVPhyl
  • Sample_id - contains all the files that were created for a particular sample in this sequence type.
• Run Specific Files - Files that summarizes output
• Pipeline information - Report metrics generated during the workflow execution

Sequence Type Specific Files

For full documentation of SNVPhyl produced files see SNVPhyl documentation.

Output files

• ST#/
  • <ST>_centroid_info.txt: A text file that states which sample was used as a reference in building the tree. If there was a tie between samples they will all be listed.
  • <ST>_cleaned_metadata.tsv: The same as the file passed with --metadata, but date will be changed to work with Microreact. Additionally, there latitude and longitude values will be added if country, state, county and city columns were present in the metadata file.
  • <ST>_phylogeneticTree.newick (SNVPhyl File): The file phylogeneticTree.newick is the maximum likelihood phylogeny generated from an alignment of SNVs extracted from the whole genomes of each input file in Newick format.
  • <ST>_snvAlignment.phy (SNVPhyl File): The file snvAlignment.phy contains an alignment of SNVs used to generate the phylogenetic tree.
  • <ST>_snvMatrix.tsv (SNVPhyl File): This file represents a pair-wise distance matrix of SNVs that passed all filtering criteria (has status valid in the snvTable.txt).
  • <ST>_vcf2core.tsv (SNVPhyl File): This file is a table of the evaluated core positions in each reference fasta sequence.
    The columns are as follows:
    • Reference name: The reference fasta sequence name. The value of all represents the sum of all sequences.
    • Total length: The total length of the fasta sequence.
    • Total invalid and excluded positions: The total number of invalid positions removed from analysis.
    • Total valid and included positions: The total number of valid positions to be evaluated.
    • Total valid and included positions in core genome: The total number of valid positions within the core genome. That is, the total positions with enough coverage to be evaluated.
    • Percentage of valid and included positions in core genome: The percent of valid and included positions within the core genome. That is 100*(valid and included positions in core)/(valid and included positions).
    • Percentage of all positions that are valid, included, and part of the core genome: The percent of valid and included core positions out of the total length. That is 100*(valid and included positions in core)/(Total length).
  • filtered_density_all.txt (SNVPhyl File):
  • filterStats.txt (SNVPhyl File): A summary of the number of SNVs filtered within in the SNV Table.
  • mappingQuality.txt: This file mappingQuality.txt describes how well the given reads mapped to the reference genome. This can help to evaluate if one genome has either too little data to be included in the analysis, or if the reference genome is very distantly related, or other potential issues.
  • new_invalid_positions.bed (SNVPhyl File):
  • phylogeneticTreeStats.txt (SNVPhyl File):

Sample Specific Files

Output files

• <sample_id>.bam (SMALT File):
• <sample_id>_sorted.bam (SAMTools File):
• <sample_id>_filtered_density.txt:
• <sample_id>_freebayes_filtered.vcf.gz (Freebayes File): A compressed Variant Call Format (VCF) file that has been filtered to remove insertions/deletions and split complex variant calls.
• <sample_id>_freebayes.vcf (Freebayes File): This Variant Call Format (VCF) file is a text format that follows the specification for this file type. In case you aren't familiar with VCF files, it usually has several header lines before the actual data; the header lines start with ##. Variants are represented in the rows, and each sample has a column with the status of a given variant. For example:

##format=VCFv4.0
##fileDate=20090805
##source=myImputationProgramV3.1
##reference=1000GenomesPilot-NCBI36
##phasing=partial
#CHROM  POS     ID        REF   ALT    QUAL  FILTER  INFO                                 FORMAT       NA00001         NA00002         
20      14370   rs6054257 G     A      29    0       NS=55;DP=255;AF=0.768;DB;H2          GT:GQ:DP:HQ  0|0:48:1:51,51  1|0:48:8:51,51  
20      13330   .         T     A      3     q10     NS=55;DP=202;AF=0.024                GT:GQ:DP:HQ  0|0:49:3:58,50  0|1:3:5:65,3    
20      1110696 rs6040355 A     G,T    67    0       NS=55;DP=276;AF=0.421,0.579;AA=T;DB  GT:GQ:DP:HQ  1|2:21:6:23,27  2|1:2:0:18,2    
20      10237   .         T     .      47    0       NS=57;DP=257;AA=T                    GT:GQ:DP:HQ  0|0:54:7:56,60  0|0:48:4:51,51  
20      123456  microsat1 G     D4,IGA 50    0       NS=55;DP=250;AA=G                    GT:GQ:DP     0/1:35:4        0/2:17:2         

Often the header lines will have some explanation about the various columns in the VCF, including the confusing looking INFO column. For more explanation of the INFO column see page 5 #8 in the VCF format documentation.

• <sample_id>_mpileup.vcf.gz (Bcftools File):

SMALT

SMALT aligns DNA sequencing reads with a reference genome. SMALT employs a hash index of short words of up to 20 nucleotides sampled at equidistant steps along the reference genome. For each sequencing read, potentially matching segments in the reference genome are identified from seed matches in the index and subsequently aligned with the read using dynamic programming.

SAMTools and BCFtools

BCFtools is a set of utilities that manipulate variant calls in the Variant Call Format (VCF) and its binary counterpart BCF. SAMtools and BCFtools are run independently of FreeBayes and are used to confirm the FreeBayes variant calls and generate base calls for non-variant positions. For more details see SAMTools

Freebayes

In PhyloPHoeNIx is run in FreeBayes is run using the haploid variant calling mode. Freebayes is a Bayesian genetic variant detector designed to find small polymorphisms, specifically SNPs (single-nucleotide polymorphisms), indels (insertions and deletions), MNPs (multi-nucleotide polymorphisms), and complex events (composite insertion and substitution events) smaller than the length of a short-read sequencing alignment.

Freebayes is haplotype-based, in the sense that it calls variants based on the literal sequences of reads aligned to a particular target, not their precise alignment. This model is a straightforward generalization of previous ones (e.g. PolyBayes, samtools, GATK) which detect or report variants based on alignments. This method avoids one of the core problems with alignment-based variant detection--- that identical sequences may have multiple possible alignments. Find out more in Freebayes' Docs.

Run Specific Files

Output files

• Directory_samplesheet.csv: This file contains the best taxa id. The number after the ":" is the NCBI assigned taxID for easy lookup.
• SNVPhyl_GRiPHin_Summary.xlsx: Like a typical PHoeNIx summary file for the samples in your sample set, but below that data the SNV Matrices produced by Phylophoenix are present. Also, states the % genome used for the analysis.
• GRiPHin_Summary.xlsx: An excel file that summaries the output of both PHoeNIx. See full explanation of columns in PHoeNIx wiki

Pipeline information

Output files

• pipeline_info/
  • Reports generated by Nextflow: execution_report.html, execution_timeline.html, execution_trace.txt and pipeline_dag.dot/pipeline_dag.svg.
  • Reports generated by the pipeline: pipeline_report.html, pipeline_report.txt and software_versions.yml. The pipeline_report* files will only be present if the --email / --email_on_fail parameter's are used when running the pipeline.
  • Reformatted samplesheet files used as input to the pipeline: samplesheet.valid.csv.

Nextflow provides excellent functionality for generating various reports relevant to the running and execution of the pipeline. This will allow you to troubleshoot errors with the running of the pipeline, and also provide you with other information such as launch commands, run times and resource usage.