Skip to content

BulkSeq Studio 0.26.6

Choose a tag to compare

@tunabirgun tunabirgun released this 07 Aug 04:32
· 65 commits to main since this release

If you produced a multi-study meta-analysis result with 0.26.5 or earlier, re-run it. The Benjamini-Hochberg correction was applied to a different family of hypotheses than the one reported. No other analysis route is affected.

Fixed

  • The multi-study meta-analysis corrected for multiplicity over the wrong family.
    run_meta_analysis.R adjusted the combined p-values across every gene and then removed
    direction-discordant genes from the called set, so the reported meta-DEGs were a
    Benjamini–Hochberg rejection set minus a subset chosen after the threshold was fixed. The
    guarantee does not transfer to such a subset, and the filter is not independent of the
    p-values: a gene with strong opposing per-study effects has both a small combined p-value and a
    high chance of being discordant, so discordant genes crowded the low tail, raised the cutoff and
    admitted null genes that then survived the filter. Measured in B19's discordance arm, which
    plants only discordant genes and therefore contains no true positive, the combination reported
    58 meta-DEGs across five runs, every one a false positive, at approximately α × (discordant
    rejections) per run. The correction restricts the family to concordant genes before adjusting;
    discordant genes keep their row and raw combined p-value but carry no adjusted value, because
    they are not being tested. Re-run, the arm reports 1 gene across five runs, and the power
    arm improves slightly because the tested family is smaller. Anyone who has run a meta-analysis
    should re-run it: the differentially expressed gene set changes.
  • The meta-volcano would have drawn discordant genes as the most significant points. With
    discordant genes carrying no adjusted p-value, NA <= 0 | !is.finite(NA) evaluates to TRUE, so
    the fit-all branch placed every one of them at the capped ceiling. The volcano now excludes
    genes with no adjusted p-value and states how many in its subtitle.
  • B19's acceptance criteria missed the defect they existed to catch. Criterion C asked only
    whether discordant genes were kept out of the called set, never what the called set contained.
    A new criterion F bounds the discordance arm's false-positive count; scored against the pre-fix
    data it fails at 5 of 5 runs with a lower confidence limit of 0.478.
  • Five release gates could not fail. check_typography.py returned 0 unconditionally and, run
    from outside the repository root, read no files while reporting "0 issues across 2 file(s)";
    build_main_tables.py returned 0 regardless of its own checks, so deleting a table body
    propagated silently into every deliverable; an early return made the article-wide placeholder
    sweep unreachable; check_b19 verified only the study-count stratum that decides the verdict,
    which let a wrong three-study count stand in the manuscript; and extract() scanned forward
    with no stop condition, so a missing table body made it return a different table's rows. Each is
    now demonstrated to fail on an injected defect.
  • B19 was not regenerable from the deposit. Its simulator sources
    workflow/scripts/run_meta_analysis.R, which lives outside article/ and was never staged. The
    archive builder now derives its external dependencies from what the deposited scripts actually
    source.

Changed

  • Benchmark archive deposited (Zenodo) as version 0.26.6, 10.5281/zenodo.21833538, under the concept DOI 10.5281/zenodo.20955660, which resolves to it. This is the first deposit carrying the corrected meta-analysis; 0.26.4 and 0.26.5 were built but never deposited, and every earlier deposit describes software with the defect.

Added

  • scripts/preflight.py runs every release gate in one command and reports which could not
    run. The manuscript and benchmark gates cannot run in CI because article/ is gitignored, so
    they only ever run when someone runs them; a gate that cannot run is reported as a failure
    rather than a pass.

Benchmark archive for this release: https://doi.org/10.5281/zenodo.21833538 (version 0.26.6). The concept DOI https://doi.org/10.5281/zenodo.20955660 always resolves to the latest version.