tsoppy input standardization - 1. Quality metrics

[danielvo]

Inputs considered so far:

• Dragen TSO500 analysis pipeline v. 2.6.2
  • the information below is based on output generated for samples sequenced with TSO500 assay version 2, but the current
  • unclear: is the Dragen server version a factor?
    • OUS assumes it does not, and both OUS and UNN use the same Dragen server version.
      • It looks like both assay v1 and assay v2 samples can now be analyzed with the same Dragen software.
    • Dragen server version [07.021.624.]3.11.2 was used when generating the output referenced in the text below
    • UNN utilizes the same Dragen server version and Dragen TSO500 analysis pipeline version
    • unclear: When analyzing samples with ICA (Illumina Cconnected Analysis), can one pick exactly the same Dragen server version and TSO500 pipeline version?
  • unclear: can the analysis output structure vary based on the sample sheet content/format choices?
    • different Pair_ID and Sample_ID values were used when generating the output referenced in the text below
    • we should aim to standardize the sample sheets across InPreD nodes either way
• Local App version 2.2.0.12
  • the sample sheet content/format (e.g., the Pair_ID value choice) can impact the analysis output structure
    • different Pair_ID and Sample_ID values were used when generating the output referenced in the text below

1. Quality metrics

1.1. Input files

Dragen TSO500 2.6.2:

• file path: Logs_Intermediates/MetricsOutput/MetricsOutput.tsv

DRAGEN TruSight Oncology 500 v2.6.2 Analysis Software - Metrics Output				
For Research Use Only. Not for use in diagnostic procedures.				
				
[Header]				
<...>	
Workflow Version	2.6.2.4			
				
[Run QC Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	Value	
<...>
				
[Analysis Status]				
	TVD0016	TVD0014		
<...>
				
[DNA Library QC Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Library QC Metrics for Small Variant Calling and TMB]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Library QC Metrics for MSI]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Library QC Metrics for CNV]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Library QC Metrics for GIS]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[RNA Library QC Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Expanded Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[RNA Expanded Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[Notes]				
Run Metrics	Run Metrics are not generated and values are reported as NA when starting analysis from FASTQ files.

Local App 2.2.0.12:

• file path: Logs_Intermediates/MetricsOutput/MetricsOutput.tsv

TruSight Oncology 500 - Metrics Output				
For Research Use Only. Not for use in diagnostic procedures.				
				
[Header]				
<...>	
Workflow Version	ruo-2.2.0.12			
				
[Run QC Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	Value	
<...>
				
[Analysis Status]				
	TVD0016	TVD0014		
<...>
				
[DNA Library QC Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Library QC Metrics for Small Variant Calling and TMB]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Library QC Metrics for MSI]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Library QC Metrics for CNV]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[DNA Expanded Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[RNA Library QC Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[RNA Expanded Metrics]				
Metric (UOM)	LSL Guideline	USL Guideline	TVD0016	TVD0014
<...>
				
[Notes]				
Run Metrics	Run Metrics are not generated and values are reported as NA when starting analysis from FASTQ files.

1.2. General observations:

• Both Dragen and Local App generate a joint metrics overview file for both run-level metrics and sample-level metrics
  • The file has the same path and name in case of both analysis pipelines
• The available metrics and other information are divided into individual sections with own headers (and varying structure)
  • General information is present in sections [Header] and [Notes]
    • The Workflow Version item under the [Header] section (see the examples below) seems to provide an identification of the analysis pipeline
  • Run-level metrics are limited to a section titled [Run QC Metrics]
  • Sample-level metrics are present in the remaining 8-9 sections
    • An HRD-related metrics section ([DNA Library QC Metrics for GIS]) is specific to the Dragen pipeline
• A single metric has a sample type prefix: RNA_PCT_Q30_BASES in case of the Dragen output
• Multiple metrics with the same name are defined for both DNA and RNA samples (MEDIAN_INSERT_SIZE, PCT_CHIMERIC_READS, TOTAL_PF_READS)

1.3. Differences between Dragen and Local App metrics output

• Metrics unique to Dragen
  • ALLELE_DOSAGE_RATIO
  • EXCESSIVE_TF (HRD samples only)
  • GENE_ABOVE_MEDIAN_CUTOFF
  • GENE_MEDIAN_COVERAGE
  • MEDIAN_TARGET_HRD_COVERAGE
  • PCT_Q30_BASES
  • PCT_SOFT_CLIPPED_BASES (one for DNA samples, one for RNA samples)
  • PCT_TARGET_50X
  • PCT_TARGET_HRD_50X (HRD samples only)
  • RNA_PCT_Q30_BASES
• A metrics unique to Local App
  • PCT_PF_UQ_READS
• Metrics with slightly different reporting between the two pipelines
  • Local App reports CONTAMINATION_P_VALUE and CONTAMINATION_SCORE, while Dragen reports only CONTAMINATION_SCORE
  • metric COVERAGE_MAD has upper specification limit (USL) of 0.210 in case of LocalApp; metricGENE_SCALED_MAD has USL of 0.134 in case of Dragen
  • MEDIAN_CV_GENE_500X with USL of 93 (%) in case of Local App vs. MEDIAN_CV_GENE_500X with USL of 0.93 in case of Dragen
• Different lower/upper specification limits (LSL/USL)
  • TOTAL_ON_TARGET_READS has LSL of 2,500,000 in case of Dragen and 9,000,000 in case of Local App
  • PCT_CHIMERIC_READS (the DNA-related one) has USL of 8.00 in case of Dragen (and is placed in the [DNA Library QC Metrics for Small Variant Calling and TMB] section); the metric has an undetermined USL in case of Local App (and is placed in the [DNA Expanded Metrics] section)
• Analysis of the same FASTQ files leads to different values for the corresponding metrics reported by Local App and Dragen
  • even after standardization, one might not want to mix Local App and Dragen outputs in the same plots and/or analyses

1.4. Metric data usage in tsoppy modules

1.4.1. Modules that would utilize the metric data

• Creation of metrics plots for the last n runs (PDF) - used for continuous quality control monitoring
• A metric summary for given DNA or RNA sample (PDF, PNG) - used in tumor board reports for individual patients

1.4.2 Suggestions for implementation

• The structure of the MetricsOutput.tsv files as produced by the Illumina pipelines is inconvenient
  • The different sections blend together information about the analysis pipeline, the sequencing run as a whole, as well as individual samples
• As in case of TSOPPI post-processing, it would be convenient to re-format the metrics output into two distinct tables:
  • a run-based table (where each line represents a sequencing run and each column represents a run-level metric)
  • a sample-based table (where each line represents a sample and each column represents a sample-level metric)
  • both tables can contain information from multiple runs (allowing plotting and testing across large run/sample cohorts)
  • parsing needs to be done in a section-aware manner, but we don't need to track the section of origin for each metric in tsoppy's output (it won't matter if a metric changes the section of origin, as in case of the PCT_CHIMERIC_READS metric)
    • all DNA-related metrics can be given prefix "DNA_", and all RNA-related metrics can be given prefix "RNA_" - this should make all metric titles unique and make it immediately clear what sample type a given metric relates to (the RNA_PCT_Q30_BASES metric does not need an extra prefix..)
  • all metrics are already reported for all samples (e.g., RNA samples are assigned value "NA" for all DNA-specific metrics) - in the same spirit, one could assign "NA" values to all Dragen- and Local App-specific metrics when post-processing files generated by the other pipeline
  • Local App's MEDIAN_CV_GENE_500X metric values could be converted to the Dragen units
  • unclear: What to do with the incompatible thresholds? Should both thresholds be shown on tsoppy plots?

[marrip]

First of all, that is a very nice summary. For process_metrics_files.py the RunCompletionStatus.xml from the sequencer is another input we need to consider for possible input classes and plots if I am not mistaken?

For the processing of the MetricsOutput.tsv we need a parser that can read this sectionized tsv format and place the relevant information into a class instance. I am thinking about two classes: one for the run specific parameters and a class for sample specific parameters. The question if we would like to group samples according to patient as well or not. The fields of the classes can be NA or None by default and only be set to a specific value if we could parse something from MetricsOutput.tsv. After parsing, we should have methods to return formatted data for different plots which can then just be plotted. Do we want to plot all the different parameters? Is there something we can skip? Should we throw an error if the user tries to mix data from dragen and LocalApp or rather warn and just separate them into 2 pdfs? Should plots be in pdf or html, both? html might allow us to make them more interactive? We could even explore to make all past runs available and default plot the last 10 but via inputs the user could choose as few or as many as they want? No need to replot every run but rather have more flexibility?

Any other output besides the plots that is necessary? Do we need the tables or are they just the base for the plots?

Regarding differences in values, if we can convert from one to the other we could make the conversion part of the parsing or we provide methods to convert whenever we need to.

Should we omit any empty rows (only NA or None) in the plots?

process_metrics_files.py needs the run id as input, could we simply parse that from RunCompletionStatus.xml?