StateTrace 0.3.1

pip install statetrace-sc==0.3.1 (import name statetrace).

0.3.0 reformatted; no code change. The CI lint job had failed ruff format --check on two
files since the v0.2.1 tag, and the release workflow's publish job depends on it, so neither
0.2.1 nor 0.3.0 reached PyPI. 0.3.1 is the first published 0.3 release; its behaviour, API and
record columns are those of 0.3.0 (31e1856, the commit installed from the git tag on
Bridges-2 for the benchmark's application stages).

Fixed

• Formatting of robustness.py and tests/test_robustness_aggregation_features.py to the
  pinned ruff (0.6.9); CI and the release workflow pass again.

StateTrace 0.2.0

pip install statetrace-sc==0.2.0 (import name statetrace).

StateTrace 0.2.0 expands reference validation and transport diagnostics
with calibrated out-of-reference detection, mapper-independent reference
reproducibility diagnostics, study-aware novelty calibration, and
complementary cell- and donor-level distribution-shift tests.

Highlights

• Improved reference validation. Reproducibility can now be evaluated
  in a frozen expression space, independently of mapper-specific latent
  coordinates. Study-aware holdout calibration accounts for variation
  across represented studies, with fallback for insufficient evidence.

• Calibrated novelty detection. kNN distance replaces centroid
  distance as the default OOD statistic following a prespecified
  simulation-based comparison. Support-deficit and combined
  kNN-support diagnostics are available for sensitivity analysis.

• Complementary shift evidence. E-distance permutation testing and
  optional scXMatch cross-match testing assess cell-level and
  donor-profile distribution differences without changing state
  classifications.

• Reference-aware caution thresholds. Recovery and uncertainty
  warnings now use held-out reference evidence rather than fixed
  thresholds where sufficient calibration data are available.

• Reproducibility and compatibility. Frozen-reference format 4 adds
  cell-level expression coordinates and donor profiles. Format 3
  references remain loadable. Configuration provenance and fallback
  decisions are recorded.

Benchmark status

Benchmark G1 was evaluated using StateTrace commit 07def17
(StateTrace-benchmarking 00_gates/_m/G1.md, 2026-10-16).

The benchmark passed insufficient-state recall (1.000) and
cross-backend status agreement (0.843). Withheld-state detection
did not meet its registered criterion (AUROC 0.652 versus the
registered comparison value of 0.722).

The withheld-state detection failure remains accepted as a
documented limitation, not a successful benchmark outcome.

Release defaults were selected using simulator-based criteria rather
than optimized against G1. Inspection of G1 identified an additional
mixed-neighborhood failure mode, motivating support-based diagnostics.
Those diagnostics did not satisfy the registered replacement rule and
are retained as post hoc sensitivity tools rather than new defaults.

The G1 results describe the evaluated commit and do not independently
validate additions made after that evaluation.

Added

• Out-of-reference scores that see a population the reference lacks when it sits between
  reference states rather than away from them (TransportConfig.ood_score): "support", the
  local-support deficit standardised against the held-out support of the assigned state
  (novelty.support_novelty), and "knn_support", the larger of the standardised kNN
  distance and the support deficit. Benchmark G1 (StateTrace-benchmarking) withheld CD4 T
  cells from a PBMC reference that keeps Tregs and CD8: every mapper absorbed them into
  Treg or CD8 with kNN and centroid distances no larger than those states' own held-out
  cells (AUROC at chance), while their mixed neighbourhoods were visible; the frozen
  expression space showed the same (CD4 cells sit closer to the Treg centroid than held-out
  Tregs do), so no distance score, in any space, sees such a state. The score the benchmark
  predeclares is chosen on the simulator by a rule fixed before it runs, as knn was.
  distance_calibration gains support_mean_holdout, support_sd_holdout,
  supportz_q95/q99_holdout and knn_support_q95/q99_holdout; cells gains
  support_novelty, knn_support_novelty and ood_novelty (the standardised score behind
  the out-of-reference call, so a benchmark ranks what the rule used). Frozen references
  without the new calibration columns fall back to the centroid z rule for the new scores,
  and route.reference["ood_fallback"] says so.
• simulate_transport(novel_chimera=...): a withheld state that takes a reference state's
  latent centroid exactly and, on a few seeded genes, a third state's expression level; the
  case of a population that shares its neighbour's programs but differs in marker genes.
• Two-sample engines (statetrace.twosample, TwoSampleConfig): the E-distance permutation
  test (native; Euclidean, as in pertpy) and Rosenbaum's cross-match test through scXMatch
  (Moeller et al., Bioinformatics 2026, doi:10.1093/bioinformatics/btag706; optional, conda
  only). TransportConfig(two_sample=...) runs them for every reference state at cell level
  (frozen cell-level space) and on donor profiles (held-out reference donors against query
  donors), over all assigned and in-reference cells; route.two_sample, two_sample.csv and a
  report section. Reported evidence only: status and STATE_COLUMNS are unchanged.
• Frozen cell-level space in ExpressionSpace (cell_sample_per_state, cell_components,
  cell_seed; build_cell_space, project_cells) and the held-out reference donor profiles
  (null_profiles); query_profiles exposes the standardised donor profiles behind
  score_query.
• simulate_transport(spread_states=...): a change in spread with the mean kept (truth
  spread, column spread_factor).
• LODOConfig(calibration_unit="study"): the novelty null of a leave-one-group-out run
  (held-out distances, their quantiles, the population thresholds) is built with one study
  held out at a time, each held-out donor's latent offset removed as for a query donor, so a
  query from a new study is judged against between-study differences an offset does not
  remove. States without enough study-held-out evidence keep the group null;
  distance_calibration["calibration_unit"] and route.reference["novelty_calibration_unit"]
  record the choice. Reproducibility still comes from the group holdout. Default "auto"
  (see Changed).
• LODOConfig(reproducibility_space="expression"): the reproducibility category is judged
  by the grouped holdout repeated with the kNN on every reference cell in the frozen
  cell-level expression space (expression.cell_coordinates), built from counts alone, so
  the same reference cells get the same call whichever mapper's latent they sit in. The
  latent call is kept as reproducibility_latent; the expression holdout adds
  reproducibility_expression and *_expression metrics. Transport reports
  ref_reproducibility_latent and ref_reproducibility_expression beside the deciding
  ref_reproducibility, and route.reference["reproducibility_space"]. Default "auto"
  (see Changed).
• Elongated states and a withheld state between two states in the simulator:
  simulate_reference / simulate_transport(state_anisotropy=, anisotropic_dims=) stretch
  each state's noise along its own random directions (drawn from a separate generator, so
  every existing seed is unchanged), and simulate_transport(novel_between={novel: (a, b, t)}) places a withheld state a fraction t of the way from a to b. With elongated
  absorbing states the centroid score loses an absorbed population that the kNN distance
  still finds (tests/test_transport.py: AUROC 0.66-0.88 centroid against 0.89-0.98 kNN over
  three seeds at t=0.35, stretch 3); the round-state grid behind the predeclared
  ood_score never showed this.
• TransportConfig(caution_calibration="holdout"): the recovery and uncertainty caution
  triggers compare the query's donors with held-out reference donors, both measured from
  StateTrace's own evidence in the mapper's latent (novelty.group_evidence_table), instead
  of fixed cut-offs (recovery below 0.8, uncertain fraction above 0.2): caution when the query
  is worse with one-sided p < caution_alpha (Fisher exact for recovery, Mann-Whitney for
  per-donor uncertain fractions) and by more than caution_margin. States seen in too few
  held-out donors keep the cut-offs (caution_basis). New state columns caution_basis,
  ref_groups_observed, cmp_recovery_rate, ref_recovery_rate, recovery_p,
  cmp_uncertain_fraction, ref_uncertain_fraction, uncertain_p and, in every mode,
  caution_reasons (also in the report); route.reference["caution_calibration"]. Default
  "holdout" (see Changed).

Changed

• Distribution name on PyPI is statetrace-sc: PyPI refuses statetrace as too similar to
  the unrelated state-trace project (separators are ignored in the comparison). The import
  name, the repository and the documentation keep statetrace; pip install statetrace-sc.
• Defaults that change results on the same data (benchmark gate G1 failed on 0.1.0: the
  status differed across mappers and the centroid novelty score lost withheld populations;
  every fix was chosen on the simulator, never on the G1 data):
  • LODOConfig.reproducibility_space defaults to "auto": the frozen cell-level expression
    space when the reference has it, else the latent. The reproducibility call no longer
    depends on which mapper's latent the reference sits in.
  • LODOConfig.calibration_unit defaults to "auto": the study-held-out null when the
    reference has at least two studies, else the donor-held-out null.
  • TransportConfig.caution_calibration defaults to "holdout": caution when the query's
    donors do measurably worse than held-out reference donors, not when a noisy rate crosses
    a fixed cut-off.
  • ValidationResult.config records the resolved values. reproducibility_space="latent",
    calibration_unit="group", caution_calibration="fixed" and ood_score="centroid"
    reproduce 0.1.0.
  • TransportConfig.ood_score defaults to "knn" (was "centroid"), set from the
    withheld-state curve on elongated states (StateTrace-benchmarking S1.3b,
    withheld_geometry_curve.csv; 240 runs, 5 seeds, StateTrace 93e7261) by a rule fixed
    on 2026-10-12, before any result of that curve. Over the G1 mappers (scANVI,
    ...