Skip to content

pycoqc-ng 3.1.0

Choose a tag to compare

@duceppemo duceppemo released this 21 Oct 00:51
· 61 commits to main since this release

pip install --upgrade pycoqc-ng

A report for runs that have not been basecalled, and a long list of
fixes found by reading the code against real Albacore, Guppy and Dorado
output rather than against the test fixtures.

Added

  • A report for runs that have not been basecalled. POD5, and a FAST5 holding
    signal only, carry no sequence, so pycoQC used to stop with an error.
    It now reports on the run itself: reads over time, time in pore, activity
    per channel and reads per barcode, with a signal summary table in place of
    the basecall one. There is deliberately no estimated base count, since
    bases can only be guessed from the signal length.
  • Pass and fail come from the passes_filtering column when there is no
    quality score to threshold on.

Fixed

  • --filter_calibration did nothing on any Guppy 2.2+ run, which names the
    column calibration_strand_genome rather than ..._genome_template.
    Recognising the name alone was not enough: the filter kept a list of the
    markers meaning "no control found", and Guppy's markers are not Albacore's,
    so the whole run would have been discarded as calibration strands.
  • Barcodes written as bare numbers, as Deepbinner writes them, were typed as
    numbers and so split across chunks, became 6.0 after a merge, and could
    not be sorted against unclassified.
  • The FASTQ and BAM converters counted elapsed time from the first read in
    the directory rather than from the start of each run, so two one hour runs
    recorded two days apart were reported as a single 50 hour run.
  • Reads missing an optional column were discarded from the read count, the
    yield and the N50.
  • Histogram bins: the shortest read fell outside the log bins, the axis ran a
    quarter of a decade past the longest read, the 1D density was drawn half a
    bin to the right, and the 2D density half a bin left and down.
  • The yield on the over time plot disagreed with the summary table, because
    bases were estimated from a sample scaled by the ratio of read counts.
  • The two mean coverage figures in one report measured different things.
  • The barcode figure's buttons took two clicks and only updated the table;
    at 96 barcodes the legend, the pie and the table were unreadable or cut
    off, in the PDF as well.
  • The converters stopped descending into a directory as soon as they found
    one matching file, so a fastq_pass holding unclassified.fastq beside
    one folder per barcode converted the unclassified reads and nothing else.
  • --fields and --include_path were accepted and ignored by
    Bam_to_seq_summary, as was --include_path by Pod5_to_seq_summary;
    --basecall_id now says it has no meaning for those formats rather than
    being silently dropped. Fast5_to_seq_summary's command line default
    left out four run level fields its Python default includes.
  • FASTQ records were collected into a dict keyed by read id, so repeated
    ids merged, but only when they fell in the same 1,000 read block.
  • A bam or fastq directory holding no reads wrote an empty summary or
    raised KeyError.
  • --config_file's help has always promised a lookup in ~/.pycoQC and
    ~/.config/pycoQC/config; nothing looked there.
  • Pass and fail were split by dropping the passing reads' index labels,
    which takes every row carrying a repeated read id: a run basecalled
    twice reported no failing reads at all.
  • Numeric barcodes came out as 6.0, and sorting their names put barcode
    10 between 1 and 2.
  • A summary with a read length but no quality score, or the reverse, was
    reported as if nothing had been basecalled instead of saying which
    column was missing.
  • Fast5_to_seq_summary left duration and passes_filtering out of its
    defaults, so a raw FAST5 report had no duration plots and nothing
    marked as failing.
  • A zero duration made every log bin edge NaN; a blank passes_filtering
    counted as a failure; a barcode too rare to be sampled showed a read
    count beside a blank N50.
  • Pass and fail are read only from the part of the path below the
    directory given to the converter, so a run kept under a directory named
    for a failed experiment is no longer a run of failing reads.
  • The FASTQ and FAST5 converters sent their workers SIGTERM on the error
    path and never waited for them, so a child that outlived the signal was
    left for Python to join at interpreter exit: the conversion raised the
    error it meant to, and the process then hung on the way out with nothing
    to say why. Cleanup now waits, and SIGKILLs whatever ignored SIGTERM, on
    every exit path rather than for three hand-listed exception types.

Packaging

  • The vendored stylesheet directory was not declared as a package, which
    setuptools warned about on every build. The files were shipped anyway,
    by the package-data globs, so no release was missing them.

Full changelog: https://github.com/duceppemo/pycoqc-ng/blob/v3.1.0/CHANGELOG.md