4. Sample identification

Overview

Sample identification is performed using the following base command:
gpid identify

Sample identification takes the following arguments:
gpid identify -i <sample directory> -r <reference directory> [-g <gene performance file> -t <thresholds file> -c <confidence support file>] [-s <species groups file>] [-o <output directory>] [-f <output formats>] [--bypass_calibration] [--overwrite_outputs] [--remove_intermediates]

Required:

• -i Sample directory containing one FASTA file per gene for the sample to identify. See Sample directory.
• -r Reference directory containing one FASTA file per gene and the corresponding BLAST databases. See Reference directory.

Required but with default file location:

• -g Gene performance file. If not specified, uses file calibration/calibration_gene_performance.csv produced by gpid calibrate. See Gene Performance.
• -t Filtering thresholds file. If not specified, uses file calibration/calibration_filtering_thresholds.csv produced by gpid calibrate. See Filtering thresholds.
• -c Confidence support file. If not specified, uses file validation/validation_confidence_support.csv produced by gpid validate. See Confidence estimates.

Optional:

• -s User-defined groups of closely related species. See Species groups.
• -o Output directory (default: identification/<sample_name>)
• -f Comma-separated output formats: csv, jpg, svg, pdf, or all (default: csv,pdf). See Output formats.
• --bypass_calibration Bypass method calibration and validation by ignoring the -g/-t/-c and using dummy calibration files instead. See Bypass method calibration and validation.
• --overwrite_outputs Allow existing output files in the output directory to be overwritten
• --remove_intermediates Do not save intermediate files; only keep final output files
• -h, --help Show this help message

Outputs
GPID represents a sample identification as a Gene Parliament that represents the percentage of genes supporting all competing identifications. By default, two files are produced:

• A table <Sample_name>_gpid.csv that summarises the Gene Parliament in tabular form and includes the estimated accuracy of identification.
• A figure <Sample_name>_gpid.pdf that summarises the Gene Parliament as a type of bar plot called a "lollipop plot". Other output file formats can be specified using the flag -f, see Output formats.

For interpretation of the results, see 5. Interpretation.

Pipeline parameters

Sample directory -i

GPID analyses one sample at a time. The sample directory needs to contain one file per gene for the sample to be identified. Each file contains a single corresponding gene sequence for the sample.

Requirements:

• Gene sequences need to be in fasta format and ending with an .FNA, .fasta or .fa suffix
• The gene files need to be named with the gene name, and nothing else than the gene name should be included before the .FNA/.fasta/.fa suffix
• Gene names need to match the gene names in the reference dataset.
• Sample names in each file need to be identical.

Example contents of sample directory
Gene1.FNA:

>Sample1
TTGGGAGAAAGGCTGCTGTGGCAGCATGTGGGTGATTATGAGGGTTCAAGAGGGAAAAAT
GGGGATATTAAGATGTTGGTTGCCACGCAGAGGTCGGGGACTGCTGCTGATAAAGTGTTT
GCCTTTTGTGTTATGGTCAGGGATAATCCCGCGGCCAATTTGAGGTCGCTAGTTGCGCTT

Gene2.FNA:

>Sample1
AAAACAAAGCAGCCCGATATTGTAAAAGCAGGTGACCCGGTTCTGCATGAACCGGCAGGA

Gene3.FNA:

>Sample1
AATTACGACCCGAATCTTAGACCGCAAGAGAAGGCGGTGGAGTATGTGCGGGCTCTTACT
GCGGCCAAATTAGAGAAGATTTTTGCAAGACCGTTTATTGGAGCAATGGATGGTCATAGA

Reference directory -r

Directory containing BLAST databases of the same genes that were retrieved for the sample of interest. Each database contains representative samples of the lineage of interest. The genes retrieved for the sample may only be a subset of the genes in the reference dataset. For example, the reference directory may contain 353 genes, of which 200 were retrieved for a sample. See 1. Reference construction.

Requirements:

• Gene sequences need to be unaligned fasta files that end with a .FNA, .fasta or .fa suffix.
• Each reference sample name needs to start with genus and species name <Genus>_<species>, followed by a unique identifier for the sample.
• Use underscores _ as separators in the sample names.
• Sample names in each file need to be identical.

Gene performance -g

File detailing gene performance, i.e. the percentage of samples correctly identified during method calibration, for all genes.
This file is automatically produced during 2. Method calibration, with the file name calibration/calibration_gene_performance.csv.

Requirements:

• Comma-separated .csv file
• First column contains gene name, exactly as for the reference and samples to be identified, e.g. Gene1
• Second column contains gene performance as a percentage, e.g. 58.82

Example gene performance file:

gene	performance
Gene1	88.89
Gene2	45.16
Gene3	62.96
...	...
Gene353	72.48

Filtering thresholds -t

File containing thresholds for gene performance, alignment filtering and parliament size.
This file is automatically produced during 2. Method calibration, with the file name calibration/calibration_filtering_thresholds.csv.

Requirements:

• Comma-separated .csv file
• First row contains variable names
• Second row contains variable thresholds

Example filtering thresholds file:

min_similarity	min_length	max_gapopens	max_mismatches	max_evalue	min_bitscore	min_gene_performance	min_parliament_size
98	100	1	5	1e-60	200	30	10

Confidence estimates -c

File containing confidence estimates depending on the percentage of genes supporting the identification.
This file is automatically produced during 3. Method validation, with the file name validation/validation_confidence_support.csv.

Requirements:

• Comma-separated .csv file
• First column contains the range of percentage of support
• Columns two to four contain the probability of the identification being correct, close or wrong for this range
  • correct: correct identification to species
  • close: correct identification to species group but not to species (if species groups were specified using -s)
  • wrong: incorrect identification to species (and species group, if specified using -s)

Example confidence estimates file:

support	correct	close	wrong
[0,20]	0	0	100
(20,40]	78.57	21.43	0
(40,60]	82.14	17.86	0
(60,80]	100	0	0
(80,100]	100	0	0

Species groups -s

File specifying for each species to which user-defined group of closely-related species it belongs, such as species complex or genus.

Requirements:

• Comma-separated .csv file
• Each row corresponds to a reference species and the corresponding species group
• First column contains genus and species name
  • Use underscores _ as separator
  • Spelling needs to be exactly as for the reference species
• Second column contains name of species group

Example species group file
File species_groups.csv:

genus_species	species_group
Entandrophragma_angolense	Group_A
Entandrophragma_congoense	Group_A
Entandrophragma_bussei	Group_B
Entandrophragma_candollei	Group_B
Entandrophragma_delevoyi	Group_B

Output formats -f

This allows to specify the formats for the output files produced by sample identification. By default, a .csv table and a figure in .pdf format are produced as outputs. The figure can also be saved in .jpg or .svg format.

The selected formats needs to be separated by commas, with no whitespace. For example, -f csv,jpg,svg.

Bypass method calibration and validation --bypass_calibration

Use --bypass_calibration to ignore the -g, -t and -c flags and instead use dummy calibration files with maximally lenient thresholds that pass all filtering steps.

WARNING: This will most likely result in considerably reduced accuracy of identification. However, doing this may be justified e.g. if a calibration dataset is not available or when conducting a first explorative analysis.

Bypassing method calibration will create the following dummy calibration files:

Dummy gene performance table:
To include all genes in the analyses, set performance to 100 for all genes.

gene	performance
Gene1	100
Gene2	100
Gene3	100
...	100
Gene353	100

Dummy filtering thresholds table:
To disable all filtering, set minimum thresholds to 0 and maximum thresholds to 99999:

min_similarity	min_length	max_gapopens	max_mismatches	max_evalue	min_bitscore	min_gene_performance	min_parliament_size
0	0	99999	99999	99999	0	0	0

Dummy confidence estimates table:
To include dummy confidence estimates, specify a single bin from 0 to 100, with NA for all categories:

range_support	probability_correct	probability_close	probability_wrong
[0,100]	NA	NA	NA

Next step

Upon completion of Sample identification, continue to 5. Interpretation.