WGS Extract v0.8.6

WGS Extract 0.8.6

This release adds the desktop app's Basic and Advanced modes and reorganises
its pages, with the CLI commands behind them.

New

• extract region writes one region (MT only, Y only, Y and MT, or the
  autosomes and X) as any of BAM, VCF and FASTA in one run.

• New settings reference_genome_library (only the reference genomes and
  the files made from them; default genomes/ in the reference library) and
  dataset_library (one folder per person or sample; the old
  genome_library is still read). ref status --dataset-library replaces
  --genome-library, which still works, and help dataset-library replaces
  help genome-library. Unset, the dataset library defaults to datasets,
  no longer genomes.

• info calculate-coverage --with-bins reports the depth bins (0, 1-3, 4-7
  and 8+ reads per chromosome) after the coverage table. A run whose two
  reports are newer than the input shows them again instead of rescanning.
  The GUI's Detailed Coverage button uses it.

• Library installs can be deleted: ref aadr, ref ybrowse,
  ref spliceai, ref alphamissense and ref pharmgkb take --delete,
  vep delete removes one release of the VEP cache, and the new
  ref microarray-targets builds (or with --delete removes) the genome-wide
  hg19 and hg38 microarray targets. On the app's Library page each Install
  button becomes a confirmed Delete once its item is installed.

• ref bootstrap also installs the AADR 2M SNP metadata and the hg19ToHg38
  chain, so microarray's AADR 2M export works on every library. A library
  bootstrapped before this shows as not bootstrapped until ref bootstrap
  runs again. The app no longer asks for a separate VEP cache folder: the
  cache is the reference library's vep folder (or config
  vep_cache_directory).

• align and realign write the BAM or CRAM index while sorting instead of
  reading the whole output again to index it, saving about 5 CPU-minutes on a
  30x genome. The records are unchanged
  (#545).

• A 30x genome on a 16 GB Apple M1 records how
  every command runs on a base Mac mini, and a whole 35x HG002 genome from
  realign to report with default settings: 7.3 hours, with SNP recall of
  99.1% and precision of 97.5% against GIAB v4.2.1
  (#549).

• Faster SNP and indel calling by default: vcf snp and vcf indel call
  contigs in parallel when --threads is above 1, as microarray already
  did; the VCF records match a single-stream run's. On a 16 GB M1 a
  whole-genome SNP call took 211 s instead of 849 s. Contigs without reads
  are skipped and small ones (decoys, ALTs, HLA) are called together, so
  references with thousands of contigs don't slow it down. --no-parallel
  keeps one stream (#549).

Fixes

• microarray reuses an earlier run's outputs only when that run's inputs
  are unchanged; it saves them beside the outputs as JSON and regenerates
  when they differ.
• realign's FASTQ extraction now shows progress while it runs: the share of the input read, then the reads written. Before, a whole-genome extraction showed 0% until it finished.
• analyze comprehensive no longer stops before variant calling when the Y
  haplogroup can't be called, for example on a low-coverage male genome. It
  records a y_haplogroup_failed warning and carries on; a rerun retries the
  Y haplogroup (#549).

Desktop app

• Basic and Advanced modes: a sidebar switch. Basic (the default) shows
  the common pages and options; Advanced shows everything, including the CLI
  features the app didn't offer before and a new Workflows page.
• Pages reorganised: FASTQ, BAM, VCF, Extract, Microarray, Ancestry and
  Library each start with an Inputs section; Align, Realign and Variant
  Calling move to Extract, and VCF annotation to the VCF page.
• Detailed Coverage shows the depth bins at the end, and just displays
  them when the run is already done.
• Extract's MT-only, Y-only, Y and MT, and Autosomes and X buttons, with
  FASTA / BAM / VCF checkboxes.
• Microarray formats in one row per vendor; Reich Lab AADR (HO v1, 1240K,
  2M), WGSE Combined v3 and the 23andMe API option.
• The Library page splits into Reference Data (databases, annotation
  data and VEP caches; Install buttons become Delete once installed) and
  Reference Genomes (the genome list, which gains Code and SNs columns).
• A new Datasets page, just above Reference Data, is the place for your
  own WGS result files: the Dataset Library folder, the Test Dataset and the
  Example Datasets.
• Two separate library folders: Dataset Library (dataset_library,
  formerly genome_library, which is still read) for your datasets, and
  Reference Genomes Library (reference_genome_library) for the
  reference genomes and the files made from them, and nothing else. New
  installs use datasets and genomes in the workspace; existing installs
  keep their folders.
• Languages: English, French, German and Brazilian Portuguese.

See the app's CHANGELOG for details.

Full changelog: v0.8.5...v0.8.6