lrp_3.0.2

Relax Longshot task execution stringency (#234)

lrp_3.0.1

Release of 3.0 pipeline (#224)

This marks the release of the 3.0.* workflow series, intended to massively increase the usability of these workflows in Terra. This release, therefore, contains substantial, breaking changes from the 2.1.* workflow series.

First, processing is now more logically divided between flowcell, sample, and cohort levels. The majority of workflows are now intended to run in series - e.g. first flowcell-level processing, followed by sample-level processing (which presumes that flowcell-level outputs are sample-level inputs). Future improvements are dependent on this assumed behavior.

Second, all top-level workflows are now expected to return outputs that are compatible with Terra’s data model. To retain control over directory structure, we provide functionality for tasks to copy files to a destination bucket and return that storage path to the data model.

Finally, we’ve made many improvements to the workflows themselves. Those changes include:

PB Sequel II/IIe:

• Updated to CCS 6.0.0 (so that Sequel II data will be compatible with Sequel IIe on-board basecalling).
• Added extracthifi tool to subset CCS reads to only rq >= 0.99 reads, so that all subsequent tasks in Sequel II and Sequel IIe CCS datasets are processed on reads in the same way.

ONT Basecalling

• Upgraded to Guppy 4.5.4
• Improved basecalling on datasets where the sequencing_summary.txt and final_summary.txt files are not present
• Added support for demultiplexing with standard barcoding kits. Because basecalling can be single-plex or multiplexed, its output can technically be 1-to-1 or 1-to-many. That severely breaks compatibility with Terra's data model. Thus, we now only return the directory to which the results are being written. We will rely on scripts that scan input directories to populate the data model instead.

Assembly

• Added Canu assembly workflows
• Added Flye assembly workflows
• Added Hifiasm assembly workflows
• Added Quast to assembly workflows for automatic assembly performance reports
• Made Quast comparison against a reference sequence optional

Variant calling

• Added variant calling subworkflows for PBCCS, PBCLR, and ONT data
• Added DeepVariant-PEPPER to ONT calling
• Parallelized most variant calling tasks
• Allow variant calling to be turned off for whole genome workflows (useful if one wants to make CNV calls on the BAMs and doesn't need the SNV/SV calls immediately).

QC

• Added workflow to verify a BAM file against a sample fingerprint
• Replace the mosdepth Docker image to one with a newer version
• Added tools to compute yield stats based on PacBio .pbi files
• Fixes to IsoSeq output filenaming

lrp_2.1.57

Added an option to merge basecalled fastqs into a single file (#217)

• Added an option to merge basecalled fastqs into a single file

• Added output section to Guppy wdl

lrp_2.1.56

update workflow exe options to use new default exe bucket now that we have our own cromwell server (#211)

lrp_2.1.55

Optionally extract uncorrected data (#207)

• Added tool to extract uncorrected reads to a separate BAM file

lrp_2.1.54

Minor processing improvements for malaria data (#206)

• Make Guppy configuration specifiable

• Search subdirectories for fast5 files to basecall

lrp_2.1.53

Restore GPU tests and add Guppy and malaria assembly workflows to Dockstore workflow list. (#205)

lrp_2.1.52

Downloader workflow tests should be run without call-caching. (#204)

Added a new Cromwell resource JSON that disables call-caching for specific workflows (i.e. downloader workflows that store no intermediate results).

lrp_2.1.51

Add nightly test status badge. (#203)

Add nightly test status badge.

lrp_2.1.50

Minor fixes to FTP download tool (#202)

• Upload to GCS should only proceed if download from FTP was successful.

• Fixed bug in logic of which workflows should have final results removed before dispatching the test workflow.