v0.19.0 — unscoreable perturbations left out, panels without target genes, co-expression metric

Install

pip install -U cell-eval2

Requires Python 3.11+. The extras are unchanged — [gpu], [gpudge], [scale] — except that [scale] now also installs pyarrow>=15.

Competition scoring is unchanged

rule_version stays 3, and the competition rule digest and the score scales are bit-identical to 0.18.0's. Bundles built under rule_version 3 score under 0.19.0, and no vcc2026 number moves. Outside the competition profile some scores do move; see below.

Breaking changes — what to rebuild

• A perturbation the reference cannot score is no longer scored (v2 full and de; minimal and anndata through delta_pearson). Sixteen metrics, and their de_deseq2_* versions, are gated on the reference alone: a perturbation with fewer than 10 reference-significant genes is left out of the aggregate instead of being given the metric's worst value. The left-out set is the same for every submission, and a prediction that fails inside the gate is still penalized. ⚠️ Rebuild any full or de baseline or ceiling built before 0.19.0; nothing detects a stale one. Warm result caches are invalidated automatically.
• Partitioned PR/ROC AUC keeps tiny p-values in order. score_cellstream, score_h5ad_manifest and score_rowstore replaced the v2 default min_nonzero with replace_zero at 1e-10, whose float32 round-trip scored any p-value below about 7e-46 as 1e-10 — below a weaker 1e-20. They now use the new preserve policy: float64, floored at the smallest positive float64. Whole-table scoring and every default are unchanged. Recompute partitioned AUC baselines and ceilings; auc_pval_floor: replace_zero with auc_pval_floor_value: 1e-10 reproduces the old numbers.
• Partial results written by 0.18.0 cannot be combined with partials written by 0.19.0.
• target_gene_map: a "*" key now means "every label" and must map to None. Rebuild any anchor or baseline built before 0.19.0 from a map with a "*" key or a None value.

Added

• Panels whose perturbations have no target gene, such as drug doses or added cytokines. A None value in target_gene_map declares that a label has no target gene, {"*": None} declares it for every label, and the CLI flag --no-target-genes sets that wildcard. A declared label excludes nothing and is never matched to a gene by name. Before, 20 of the 54 full metrics raised on such a panel; now all 54 score it. Gene panels are unchanged.
• Single-cell co-expression metrics, opt-in and unscored: cell_cov_ccc, cell_cov_corr and cell_cov_eig_ratio, in the new cell profile. They compare the real and predicted gene–gene correlation structure across single cells, which no pseudobulk metric can see. They need raw cells, so only in-memory scoring computes them; the streaming and partitioned scorers refuse them.
• A ground-truth DE cache for score_cellstream, off by default. With ground_truth_cache=<root>, every prediction scored against the same ground truth reuses one real-DE table, keyed by the archive's content checksum and every setting that changes the table. cell-eval2 cache inspect validates an entry, and cell-eval2 cache doctor checks a cache root.
• de.auc_pval_floor: preserve, the policy partitioned scoring now uses.

Changed

• score_cellstream(..., write_de=True) writes its DE tables through a pinned Parquet writer, in bounded memory, and publishes its five outputs together, rolling back on any error. Rows, values and columns are the same as 0.18.0's; the file bytes differ.

Fixed

• partition_inmem.score_piece(de_out=...) validates piece_id the way partial_out does.
• The README and tutorial score_cellstream examples now run as written.

Credits

Chris Carpenter contributed the single-cell co-expression metrics, following a specification by Hani Goodarzi. Abhinav Adduri found and fixed the partitioned AUC p-value floor. Joseph Min asked for panels without target genes. Beatrice Bevilacqua raised the problem of perturbations the reference cannot score. Nick Youngblut designed the ground-truth DE cache and the DE-table output contract.

Note on issue numbers

Issue and pull-request numbers in CHANGELOG.md refer to the internal development repository this tree was copied from, not to this one.

v0.18.0 — DE tables from score_cellstream, and bundles that survive a release

Install

pip install -U cell-eval2

Requires Python 3.11+. The extras are unchanged since 0.16.0: [gpu], [gpudge], [scale].

Scoring is unchanged

rule_version stays 3, and the competition rule digest is bit-identical to 0.16.0's. No scored number moves.

A release no longer invalidates a validation bundle. cell_eval2_version is now recorded in every artifact and no longer compared on the bundle path, so bundles built under rule_version 3 — at 0.15.0 or 0.16.0 — score under 0.18.0. The stamp is still validated: a missing, empty or non-string value fails closed. The rule itself is still enforced: a bundle built under a different rule_version is refused, as before. BREAKING for direct callers only: AnchorExpect loses its version field. The loose --baseline-agg path still compares the version, and the comparison can still be waived.

Added

• score_cellstream(..., write_de=True) writes the gene-level DE tables, de_real.parquet and de_pred.parquet, next to the score frames. It is off by default and requires outdir. It uses the same writer as the in-memory --write-degenes path, so both paths emit one schema.
• The jackknife's per-gene decomposition, opt-in (per_gene_jk=True). Nothing requests it by default, and the scalar correction is bit-identical whichever route builds it.

Fixed

• With control_source="real" on a CPU DE backend, a duplicated gene axis is now refused with a ValueError naming the duplicated labels. Before, the error type depended on the installed anndata: 0.13.3 raises pandas.errors.InvalidIndexError where 0.13.2 raised ValueError.

Why there is no 0.17.0

The development tree carried the version 0.17.0 for several weeks before this release. Builds from it stamped that number, even though they lacked some of these changes. The release therefore takes 0.18.0, so that no unreleased build shares its number. An artifact stamped 0.17.0 came from an unreleased development build.

Credits

Chris Carpenter (@cachris1) reported the missing DE-table export from score_cellstream and contributed the fix. Nick Youngblut (@nick-youngblut) wrote the duplicated-gene-axis guard.

Note on issue numbers

Issue and pull-request numbers in CHANGELOG.md refer to the internal development repository this tree was copied from, not to this one.

v0.16.0 — first public release

First public release of cell-eval2 — a GPU-accelerated reimplementation of
cell-eval, Arc Institute's suite for evaluating
perturbation-prediction models at single-cell resolution.

Install

pip install cell-eval2

Requires Python 3.11+. The base install is CPU-only and pulls nothing private.

extra	adds	for
[gpu]	cupy, nvidia-nvcomp-cu12	CUDA pseudobulk / distance / rank kernels
[gpudge]	gpudge	GPU differential expression
[scale]	cellstream	scoring archives out of core

What's here

• 54 metrics in the full profile (48 of them scored), in four classes: expression error,
  perturbation discrimination, signed-effect geometry, and differential expression.
• Profiles and presets rather than metric-by-metric selection — --profile de,
  --preset vcc2026, and a cell-eval-0.7.6 preset that reproduces upstream cell-eval.
• GPU where it pays, with a CPU path for everything.
• Out-of-core scoring for datasets that do not fit in memory.
• v1 ↔ v2 naming with cell_eval2.compat for byte-for-byte parity against upstream
  cell-eval.
• Every run stamps a config digest; the competition path additionally pins a rule digest that a
  submission is checked against exactly.

Start with the tutorial; the metrics reference has the math
for every metric.

The 2026 Virtual Cell Challenge

docs/vcc2026_metrics/ carries the specification entrants are scored under — the full document
and an abridged reference brief, both as Markdown and PDF. --preset vcc2026 reproduces the
scored configuration.

⚠️ If you hold validation bundles built at 0.15.0, they will refuse a run at 0.16.0.
cell_eval2_version is the first entry in the bundle's submission peers and is compared exactly,
fail-closed, with no waiver. Nothing about the scoring rule changed — rule_version stays 3 and
the rule digest is bit-identical — so bundles rebuilt at 0.16.0 reproduce every number and differ
only in the stamped version. Those rebuilt bundles follow shortly.

Why 0.16.0 is the first published version

Every earlier version was unpublishable regardless of repository visibility: the [scale] extra
carried a direct reference to a private git repository, and PyPI rejects any distribution whose
requires_dist contains one, extras included. That reference is now the public cellstream
package.

Credits

The Spearman–Brown data ceiling — the split-half reliability estimate that bounds what any
submission can score on a finite panel — was implemented by Leon Hafner, from an idea proposed
by Beatrice Bevilacqua (@beabevi).

Jeremy Sullivan (@sullivanj91) found, quantified and fixed
the perturbation-discrimination target-gene exclusion that silently did nothing on guide-level
labels — under that defect a submission predicting nothing but its own on-target knockdown
outscored real ones. Abhinav Adduri (@abhinadduri)
contributed the model-conditioned direction metric and the GPU pdex fork the cell-eval-0.7.6
preset pins. Chris Carpenter contributed --write-degenes.

AUTHORS lists these in full alongside everyone who reported, diagnosed or proposed a change —
several of which changed a scored number or a documented rule.

Note on issue numbers

Issue and pull-request numbers in CHANGELOG.md and AUTHORS refer to the internal development
repository this tree was copied from, not to this one.