Numeric handoffs for PBMC 3k and PBMC 8k, and fix find_all_markers

[shanikawm]

Closes the last gap the fresh-install audit left open: tutorials 1 and 2
were the only two in the suite still compared entirely by eye. Every R panel in
the PBMC 3k walkthrough links a canonical satijalab.org image, so nothing
failed if the numbers behind those pictures drifted.

Both verify scripts now write a numeric handoff, and both tutorials gained a
--report. Neither side is pinned to the other — each runs its own
pipeline from the same 10x bytes, which is what makes the comparison worth
anything. (pbmc3k_dimreduc_verify.R is the opposite experiment: it does pin
Python's cells and features, to isolate the post-PCA machinery.)

Per-cell rows are keyed by barcode and per-gene rows by symbol, and cluster
comparisons match the two partitions one-to-one on overlap before scoring —
cluster numbering is arbitrary in both tools, so without that a pure
relabelling reads as total disagreement.

PBMC 3k, end to end

	shanuz vs Seurat 5.5.1
Cells surviving QC	the same 2,638 barcodes; nCount/nFeature exact, percent.mt 5.3e-15
VST per gene (13,714)	mean 4.8e-14 · variance 1.6e-11 · variance.standardized 2.6e-2 relative
Variable features	1,998 of 2,000 shared, rank Spearman 0.9999
PCA (the 10 clustering dims)	matched |r| 0.9988, min 0.9946, no reordering
kNN graph	52,760 on both (2,638 × 20)
Clusters at res 0.5	8 vs 9 — ARI 0.938, 2,554/2,638 cells agree
Markers, where cells match exactly	identical gene sets (151/151, 242/242), avg_log2FC to 4.9e-15

PBMC 8k, both stages

	shanuz vs Seurat 5.5.1
Cells surviving QC	the same 7,475 barcodes
Stage 1 global clusters	13 vs 12 — ARI 0.977, 7,341/7,475 agree
T/NK compartment	Jaccard 0.9991 — 4,631 of 4,635 are the same barcodes
Stage 2 subclusters	12 vs 11 — ARI 0.916 on the shared cells
T/NK subset label, per cell	0.9821; the four sizes agree within 25 cells

The compartment Jaccard is the load-bearing number, and the reason the handoff
dumps barcodes rather than counts. Everything in stage two is conditioned on
which cells stage one selected, so a compartment of the right size drawn from
the wrong clusters would still produce subclusters, still produce a marker
table, and pass a count check while making every later number incomparable.

The defect this found

find_all_markers was missing Seurat's return.thresh = 1e-2 and returned
rows Seurat does not. The handoff pinned it the decisive way: two PBMC 3k
clusters came out with identical cell membership on both sides, and on those
two shanuz returned 190 and 383 genes against Seurat's 151 and 242 — every
extra row a gene the two agreed about numerically (max avg_log2FC difference
4.9e-15) and that Seurat simply does not return. Applying the same filter
reproduces Seurat's gene sets exactly: 151/151 and 242/242.

Two more fixes in the same function:

• Rows are now ordered by p_val ascending then avg_log2FC descending,
  matching order(gde$p_val, -gde[, 2]). Not cosmetic — Wilcoxon p-values tie
  at exactly 0 for the strongest markers (40–302 genes per cluster here), so
  "the top N markers", which every tutorial prints, was decided by incoming row
  order.
• Cluster labels iterate numerically, so eleven or more clusters no longer come
  back as 0, 1, 10, 11, 2, … Invisible below eleven, which is why every tutorial
  in the suite missed it.

return_thresh=None restores the old unfiltered table.

Stale docs corrected

tutorials/README.md recorded "Clusters at resolution 0.5: 9 ✅" and "All 6
canonical cell types recovered … DC ✅"; the vignette said "9 clusters in
both R and Python". shanuz finds 8, and the cluster Seurat has that it does
not is a 32-cell dendritic population whose cells land — all 32 — in
shanuz's CD14+ Mono cluster. The provenance is now traced in the vignette from
the LOESS fit through the 2 swapped variable features and the 286 moved SNN
edges to that one boundary.

PBMC 8k runs the opposite way: Seurat's 100-cell cluster 11 holds both
Platelet and DC and shanuz splits it in two. Same borderline population, other
side of the threshold — so neither run is uniformly finer, and the docs now say
so rather than implying a verdict.

Also

• Both R references pin nn.method = "rann" — the annoy trap that cost
  pbmc3k_objects_verify.R a false negative of 182 SNN edges. On PBMC 8k it is
  worse than cosmetic: the graph decides the global clusters, which decide which
  cells enter the subclustering stage.
• The reports now name unmatched clusters and say where the other run put
  their cells, instead of letting the Hungarian matching drop them off the table.
• A latent lexicographic sort in _assign_cell_types fixed to match the R port.

Verification

• 824 passed, 25 skipped (806 before) — 18 new tests
• Every new guard mutation-tested. One was decorative on the first attempt:
  the tie-break test survived removing the tie-break, because its fixture
  produced no ties. Rebuilt around six perfectly-separating markers (which share
  a U statistic and so a p-value) with an anti-vacuity assertion; mutation now
  caught.
• ruff check shanuz 72 and whole-repo 222, both unchanged from main
  measured with the branch stashed.
• All R-side and Python-side figures for both tutorials regenerated.

🤖 Generated with Claude Code