Skip to content

Scan four resolutions in the guided tutorial, and band the result - #97

Merged
shanikawm merged 1 commit into
mainfrom
feat/resolution-stability
Aug 7, 2026
Merged

Scan four resolutions in the guided tutorial, and band the result#97
shanikawm merged 1 commit into
mainfrom
feat/resolution-stability

Conversation

@shanikawm

Copy link
Copy Markdown
Contributor

Closes the last item from the domain-expert review — comment 4, "I typically
run 3 resolutions at the same time and compare them before deciding which one to
use for downstream analysis."

#93 gave find_clusters the vector form. This uses it.

The sweep

Both sides now run [0.4, 0.8, 1.2, 0.5]find_clusters(pbmc, resolution=[...]) and FindClusters(pbmc, resolution = c(...)) — and every
resolution is scored against R:

resolution truecell Seurat ARI concordance
0.4 9 9 0.8958 0.9602
0.5 8 9 0.8987 0.9549
0.8 11 11 0.8264 0.9174
1.2 12 12 0.7995 0.8647

Two things a single resolution could not show. The cluster count matches
exactly at 0.4, 0.8 and 1.2
— so the 8-vs-9 split this tutorial has always led
with is specific to resolution 0.5, not a standing property of the port. And
agreement falls as resolution rises, which is what finer partitions should do:
more boundaries, more chances for two Louvain runs to land in different optima.

0.5 is given last on purpose. Seurat leaves the object on the last
resolution in the sequence, and UMAP, the markers, the annotation and the
handoff CSV are all written against it. Verified rather than assumed: the
tutorial's four outputs are byte-identical to what main produces. A test
pins the ordering, since reordering the list re-points all of them silently.

Found while doing it: a stale headline number

The tutorial documented ARI 0.938 while measuring 0.899 — across six files
(tutorials/README.md ×2, pbmc3k_tutorial.md ×2, docs/fidelity.md,
docs/quickstart.md), with the concordance figures stale to match: 2,554/2,638
cells and 0.968, against a measured 2,519 and 0.955.

I checked whether this PR caused it. It did not — R's vector-form FindClusters
produces a partition identical to the single 0.5 call, confirmed by running
the main script and diffing per cell.

The likely cause is the graph fixes in #67#71 moving cells between clusters —
the same drift the DE tutorial's deseq2 top50 band did catch at the time
(25 → 22). This tutorial had no band on its headline number, so it went stale in
six documents instead of failing once. Every swept resolution now carries a
declared band, checked by --report.

The [0.9.0] changelog entry quoting 0.938 is left alone — it records what was
measured then.

Verification

Five mutants, all caught: reordering the sweep so it no longer ends on 0.5;
desynchronising the two languages' resolution lists; dropping a band; widening
one to vacuity; and reverting R to a single-resolution call. The band check was
separately confirmed to fire by narrowing one until it failed.

1179 passed, 25 skipped. mypy at its 4-error baseline. --report exits zero.
A changed ridge-plot PNG was reverted as rendering churn, the underlying
clusters having been verified identical.

Also

Generalises the CSV-precision lint from #94: dtype=str parses no floats, so it
cannot misparse one. That was a per-file exemption for the DE hex reader and is
now a rule — which is what caught this PR's own new reads.

🤖 Generated with Claude Code

Choosing a resolution means running a few and comparing them -- the reviewer's
own description of her practice -- so a fidelity claim pinned to one setting
says less than it appears to. Both sides now use the vector idiom and every
resolution is scored against R:

  resolution  truecell  Seurat     ARI
        0.4          9       9  0.8958
        0.5          8       9  0.8987
        0.8         11      11  0.8264
        1.2         12      12  0.7995

Two things read off that. The cluster COUNT matches exactly at 0.4, 0.8 and
1.2, so the 8-vs-9 split this tutorial has always described is specific to
resolution 0.5 rather than a standing property of the port. And agreement falls
as resolution rises, which is what finer partitions should do -- more
boundaries, more chances for two Louvain runs to land in different optima.

0.5 is given last on purpose: Seurat leaves the object on the last resolution
in the sequence, and UMAP, the markers, the annotation and the handoff CSV are
all written against it. The tutorial's four outputs are byte-identical to what
main produces, checked by running both. A test pins the ordering, because
reordering the list re-points all of them silently.

Found while doing it: the tutorial documented ARI 0.938 while measuring 0.899,
across six files, with the concordance figures stale to match (2,554/2,638 and
0.968 against 2,519 and 0.955). The likely cause is the graph fixes in #67-#71
moving cells between clusters -- the same drift the DE tutorial's deseq2 band
caught at the time, 25 -> 22. This tutorial had no band on its headline number,
so it went stale in six documents instead of failing once.

Confirmed that predates this change rather than being caused by it: R's
vector-form FindClusters gives a partition identical to the single 0.5 call.
The 0.9.0 changelog entry quoting 0.938 is left alone, being a record of what
was measured then.

Every swept resolution now carries a declared band, checked by --report. Five
mutants, all caught -- reordering the sweep, desynchronising the two languages'
resolution lists, dropping a band, widening one to vacuity, and reverting R to
a single call.

Also generalises the CSV-precision lint from #94: `dtype=str` parses no floats,
so it cannot misparse one. That was a per-file exemption for the DE hex reader
and is now a rule, which is what caught this commit's own new reads.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@shanikawm
shanikawm merged commit 6dabd0e into main Aug 7, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant