v0.6.0: Native Mac pseudobulk and bounded Metal buffers

ExtremaRank v0.6 adds exact integer pseudobulk optimization in the native R package. The default CPU kernel scans borrowed dense/CSC/CSR/triplet count views once, and stable integer tuple grouping reduces metadata allocation. Mac users can explicitly select backend="metal" with bounded shared input staging and two-limb integer carry/overflow checks. No Python bridge is used.

On an Apple M4 Max, complete pseudobulk calls were 3.03× / 7.10× faster than released v0.5 for fixed 100k / 250k-cell sparse inputs, with 22.1% / 24.5% lower process peak RSS. Existing Kang real counts were 9.74× faster with 32.9% lower peak RSS. All timed outputs matched an independent Matrix reference hash. These figures apply to the stated task and machine, not arbitrary GPU models or the exact ranking search.

Bounded Metal buffers fell by 75.4% / 88.4% versus whole-input staging in the synthetic cases, and 74.5% on Kang. Requested buffer bytes are distinct from total process memory; Apple Silicon unified memory is shared. Metal did not consistently outperform the new CPU pipeline, so auto selects native CPU. System GPU utilization was not measured.

Local validation: R CMD check reports OK with no errors/warnings/notes; 13,568 aggregate count values were compared across native and actually executed Metal results, representing 91,760 input positions in repeated matrix cases. Separate carry, 53/64-bit overflow and HDF5 block tests also passed. The CI-built Mac arm64 binary was unpacked and actually executed Metal on the local Mac. Existing native proof/refit regression tests also passed. Native certificates and observed model refits retain their existing scopes.

Windows and Linux continue to use native CPU. A Windows comparison runner is included for the user's later hardware validation. CUDA execution and NVIDIA GPU performance are not implemented or validated in this release.

• Mac 使用与 Windows 复现
• Measured results and all raw timing/memory records
• R-only installer

The installer verifies SHA-256 and chooses the matching R 4.6 Windows/macOS binary, otherwise the small C++17 source package. GitHub’s tagged Source code zip/tar downloads include the complete code, public benchmark data, data attribution and evidence. RELEASE_MANIFEST.json lists per-file hashes. See SHA256SUMS for the native R packages and wheel.

Final commit: bcdbe94dbde34d748f4834cc85962dadc1bf78a4. Native R / Windows / both Mac architectures / Linux / R version / Bioconductor checks and independent exactness checks passed on this commit.

ExtremaRank v0.5.0 — practical native R workflows

ExtremaRank v0.5.0 improves native R sample/donor-deletion audits for practical omics workflows. It runs directly in R with native C++ exact arithmetic and optional Bioconductor models; no Python bridge is used.

• Explicit planned/evaluated/failed/not-run coverage and correct zero-refit/partial statuses.
• Candidate membership/sign changes, actual unit witnesses, observed rank ranges and selection fractions.
• Declared model contrasts, whole-site/batch deletion and custom repeated-observation models.
• Disk/summary score modes, self-contained checksum exports, resumable model fits and reproducible PSOCK workers.
• Delayed/HDF5 row-block pseudobulk, numeric storage guards, preflight counts, summaries, plots and an executable R vignette.
• Cached exact value orders and a stronger conditional Welch variance floor; optional influence traversal preserves exact decisions.

Validation: local R CMD check has zero errors, warnings or notes; 439,536 independent Fraction checks have zero failures. Twelve native real-data analyses and five fitted-model analyses preserve the frozen v0.4 results. Serial and parallel stochastic tests match. Full coverage and failures are retained in results/v05.

Native certificates apply to frozen inputs and declared deletion budgets. Fitted-model results describe observed executed refits and do not certify unseen deletions. Memory estimates are not peak RAM guarantees. Data/source attribution remains in DATA_LICENSE.md.

R installation (restart R before upgrading an already loaded native package):

source("https://raw.githubusercontent.com/heise3/ExtremaRank/v0.5.0/r/install_extremarank.R")
install_extremarank()
library(extremarank)

R 4.6 binaries target Windows x86_64 and CRAN macOS R.framework arm64/x86_64. Other supported R versions and Homebrew R use the small source archive and a C++17 compiler. SHA256SUMS records every release asset. The full source archive additionally includes reproducible benchmarks, prior outputs and attributed example data.

Chinese v0.5 guide · Validation records · Welch bound derivation

All 12 CI jobs passed at commit 30325ae3544c2b594589dcc51660fa3b9ae68f13: native R / eight jobs, Python / four jobs. Core R package checks on Linux, Windows and both Mac architectures have Status: OK. The model job requires every optional dependency and explicitly ran HDF5-backed aggregation and serial/PSOCK equality tests.

ExtremaRank v0.4.0: native R package

ExtremaRank now runs directly in R through a standalone native R/C++ package. No Python runtime, reticulate or command-line bridge is required. Existing Python algorithms remain available.

install.packages(c("Rcpp", "BH", "Matrix", "digest"))
install.packages("https://github.com/heise3/ExtremaRank/releases/download/v0.4.0/extremarank_0.4.0.tar.gz", repos = NULL, type = "source")
library(extremarank)

The R source archive is about 41 KiB and installs without the large benchmark dataset archive. Source installation requires a C++17 compiler.

• Native paired and independent Welch audits, individual membership/sign certificates, conservative minimum deletion intervals, conditional exact extrema and one-based R witnesses.
• Dense matrices, sparse Matrix and SummarizedExperiment input, sparse donor pseudobulk and SingleCellExperiment aggregation.
• Direct R limma, limma-voom, edgeR and DESeq2 refits with frozen covariates/universes, streamed scenario caps and preserved failed fits.
• RDS and usable TSV exports, including exact strings, metadata and raw invalid scores.

Validation: local R CMD check Status OK (zero errors/warnings/notes); 1,004 cases and 439,536 independent exact checks; 183 real complete-feature ranks matched; 4,347,470 donor-pseudobulk sums matched. Standard R-only tests cover frozen Fraction golden cases, complete subsets, search limits, extreme floats, experiment objects and model integration.

Native certificates concern ranking membership/effect signs on frozen inputs. R model refits are observed sensitivity and do not certify unseen deletions. Real failed Kang edgeR and DESeq2 scenarios remain recorded without result-based repairs. CPTAC deletion examples concern technical measurement runs.

Native R guide · 中文原生 R 指南 · Validation

SHA256SUMS records the R source package, full repository source archive and Python wheel. Third-party example data retain their source terms; see DATA_LICENSE.md.

Final commit 1ed6a311b748545872558e9fafebebca074a9b5c: all eight GitHub jobs passed, including native R builds/checks on Windows, macOS and Linux and direct Bioconductor model/experiment tests. Native R CI · Regression CI.

ExtremaRank v0.3.0: candidate certificates and practical omics workflows

ExtremaRank v0.3.0 extends sample-deletion robustness auditing with individual candidate certificates and practical omics workflows.

• Certify each queried feature's Top-K membership and mean-effect sign, even when the whole shortlist changes.
• Bound minimum deletion changes; label a minimum exact only after excluding every smaller deletion count.
• Use safe conditional Welch pruning with exact rational comparisons and retained enumeration reference.
• Execute limma, limma-voom, edgeR and DESeq2 refits with explicit covariates, pairing, fixed filters and preserved failures.
• Aggregate dense, 10X or optional H5AD raw counts by donor, condition and cell type, with explicit raw layers and technical replicate declarations.
• Reproduce fixed real single-cell, technical protein and biological lipid examples, plus stable and difficult search cases.

Validation: all 59 local tests passed including optional H5AD and four R adapters; 6,000 native design/direction problems and 1,434,768 independent property/enclosure checks had zero failures. Real witnesses matched 183 full-feature Fraction rankings, pseudobulk matched 4,347,470 independent R sums, and external model scores matched 300 R-exported Top-K sets. GitHub CI covers core Python 3.10/3.12/3.14 and optional H5AD; R integrations were executed locally.

A useful real result: Nutrimouse upward Top-5 is preserved by every single-mouse deletion but changes under a two-mouse witness, giving exact minimum change 2. Four of five original candidates remain certified through budget 2.

Limits remain explicit. Favorable synthetic pruning is not a worst-case speed guarantee. Native certificates concern frozen rankings, not FDR or biological truth. A frozen Kang edgeR signed-sqrt-QL-F baseline contains tiny negative raw F values and remains NOT_EVALUABLE; its raw rows and failed scenarios are preserved without clipping or feature removal. Protein measurements are technical runs; lipid measurements are percentages.

The wheel contains the tool and packaged R adapter, with no base Python dependencies or large datasets. The source archive contains pinned public raw/prepared inputs, contracts, executed outputs and a per-file manifest. SHA256SUMS verifies both downloads. Third-party data retain their original terms; see DATA_LICENSE.md.

中文说明: https://github.com/heise3/ExtremaRank/blob/v0.3.0/docs/V03_CN.md
Usage and proof: https://github.com/heise3/ExtremaRank/blob/v0.3.0/docs/ROBUSTNESS.md
Model/single-cell guide: https://github.com/heise3/ExtremaRank/blob/v0.3.0/docs/REFITS.md

ExtremaRank v0.2.0: paired studies, Welch and external model refits

ExtremaRank v0.2 broadens candidate-list sensitivity checks to paired studies, independent two-group comparisons, and supplied external-model refits. Study reports identify actual omitted units, candidate exits/entries, and rank changes across every feasible delete-one scenario.

• Keep the original exact paired two-family extrema and shared Top-K audit.
• Add exact Welch ranking comparisons with bounded exhaustive shared-deletion search.
• Add strict matrix/metadata joins, explicit contrasts, preparation/exclusion records, and offline diagnostic reports.
• Compare full-feature external refit tables with OBSERVED_STABLE / OBSERVED_CHANGED; an executed limma moderated-t exporter is included.

Validation: 44 tests pass locally on Python 3.12 and 3.14; 2,000 independent Welch problems pass 91,227 rank comparisons and 6,000 complete status checks. The public 38-sample / 3,051-feature Golub matrix passes 117 full-feature Fraction ranking checks. The limma adapter passes 117 Top-20 matches against R. A clean wheel installation runs all three workflows with only Python and pip installed.

Native guarantees apply to frozen finite binary64 inputs and the stated statistic. Welch does not inherit the paired theorem's speedup and can be unresolved at its limit. External comparisons cover only supplied scenarios and do not certify unobserved deletions. Proteomics/metabolomics portability is not real cross-omics validation; no FDR, clinical efficacy, prediction accuracy, or global priority claim is added.

See docs/STUDY.md and docs/UPGRADE_CN.md for formats, commands, examples and scope. Third-party source attribution and LGPL material are retained separately from the MIT software. Assets contain the full reproducibility bundle, a dependency-free wheel, and SHA256 checksums.

ExtremaRank v0.1.0

ExtremaRank computes exact worst-case studentized scores and audits whether paired-omics Top-K feature sets change after shared donor deletion. Both the mean and variance are recomputed on frozen paired effects.

Included

• Conditional two-family extremum theorem, full proof and exact binary64 integer arithmetic.
• Dependency-free Python core and CSV/TSV/gzip CLI; up/down/absolute ranking.
• Exact gene envelopes, donor witnesses, and CERTIFIED / REFUTED / UNRESOLVED shared-set outcomes.
• Two provenance-checked public RNA-seq studies, complete paired inputs and executed results.
• Independent 200,000-case scalar check, 30 unit/interface tests, and shared-search ablations.
• Chinese/English documentation and an offline interactive report.

Download and run

Download and extract the source ZIP for the complete offline reproduction package. From its extremarank directory:

python -m pip install .
extremarank examples/variance_reversal.tsv --budget 1 --top-k 1 --output my-audit

The wheel installs the software alone. SHA256SUMS covers the source ZIP and wheel.

Executed evidence and scope

The 14-pair / 13,328-gene and 18-pair / 16,597-gene studies showed 22.24x / 55.29x matched speed versus exact enumeration of the same scalar target. Full public CLI execution including writes took 2.85 s / 3.94 s locally. Timings depend on hardware; they are not comparisons against approximate DE methods.

This is research software for input sensitivity of frozen paired effects and ordinary paired-t rankings. It does not estimate FDR, certify moderated-t/covariate-adjusted models, establish biomarker truth, or prove clinical benefit. Full shared-set search can remain exponential. The specific technical contribution is documented against the closest verified methods; global priority and peer-reviewed novelty have not been established.

Code: MIT. Public-data attribution and source hashes are in data/.