Scan four resolutions in the guided tutorial, and band the result - #97
Merged
Conversation
Choosing a resolution means running a few and comparing them -- the reviewer's
own description of her practice -- so a fidelity claim pinned to one setting
says less than it appears to. Both sides now use the vector idiom and every
resolution is scored against R:
resolution truecell Seurat ARI
0.4 9 9 0.8958
0.5 8 9 0.8987
0.8 11 11 0.8264
1.2 12 12 0.7995
Two things read off that. The cluster COUNT matches exactly at 0.4, 0.8 and
1.2, so the 8-vs-9 split this tutorial has always described is specific to
resolution 0.5 rather than a standing property of the port. And agreement falls
as resolution rises, which is what finer partitions should do -- more
boundaries, more chances for two Louvain runs to land in different optima.
0.5 is given last on purpose: Seurat leaves the object on the last resolution
in the sequence, and UMAP, the markers, the annotation and the handoff CSV are
all written against it. The tutorial's four outputs are byte-identical to what
main produces, checked by running both. A test pins the ordering, because
reordering the list re-points all of them silently.
Found while doing it: the tutorial documented ARI 0.938 while measuring 0.899,
across six files, with the concordance figures stale to match (2,554/2,638 and
0.968 against 2,519 and 0.955). The likely cause is the graph fixes in #67-#71
moving cells between clusters -- the same drift the DE tutorial's deseq2 band
caught at the time, 25 -> 22. This tutorial had no band on its headline number,
so it went stale in six documents instead of failing once.
Confirmed that predates this change rather than being caused by it: R's
vector-form FindClusters gives a partition identical to the single 0.5 call.
The 0.9.0 changelog entry quoting 0.938 is left alone, being a record of what
was measured then.
Every swept resolution now carries a declared band, checked by --report. Five
mutants, all caught -- reordering the sweep, desynchronising the two languages'
resolution lists, dropping a band, widening one to vacuity, and reverting R to
a single call.
Also generalises the CSV-precision lint from #94: `dtype=str` parses no floats,
so it cannot misparse one. That was a per-file exemption for the DE hex reader
and is now a rule, which is what caught this commit's own new reads.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the last item from the domain-expert review — comment 4, "I typically
run 3 resolutions at the same time and compare them before deciding which one to
use for downstream analysis."
#93 gave
find_clustersthe vector form. This uses it.The sweep
Both sides now run
[0.4, 0.8, 1.2, 0.5]—find_clusters(pbmc, resolution=[...])andFindClusters(pbmc, resolution = c(...))— and everyresolution is scored against R:
Two things a single resolution could not show. The cluster count matches
exactly at 0.4, 0.8 and 1.2 — so the 8-vs-9 split this tutorial has always led
with is specific to resolution 0.5, not a standing property of the port. And
agreement falls as resolution rises, which is what finer partitions should do:
more boundaries, more chances for two Louvain runs to land in different optima.
0.5 is given last on purpose. Seurat leaves the object on the last
resolution in the sequence, and UMAP, the markers, the annotation and the
handoff CSV are all written against it. Verified rather than assumed: the
tutorial's four outputs are byte-identical to what
mainproduces. A testpins the ordering, since reordering the list re-points all of them silently.
Found while doing it: a stale headline number
The tutorial documented ARI 0.938 while measuring 0.899 — across six files
(
tutorials/README.md×2,pbmc3k_tutorial.md×2,docs/fidelity.md,docs/quickstart.md), with the concordance figures stale to match: 2,554/2,638cells and 0.968, against a measured 2,519 and 0.955.
I checked whether this PR caused it. It did not — R's vector-form
FindClustersproduces a partition identical to the single 0.5 call, confirmed by running
the
mainscript and diffing per cell.The likely cause is the graph fixes in #67–#71 moving cells between clusters —
the same drift the DE tutorial's
deseq2 top50band did catch at the time(25 → 22). This tutorial had no band on its headline number, so it went stale in
six documents instead of failing once. Every swept resolution now carries a
declared band, checked by
--report.The
[0.9.0]changelog entry quoting 0.938 is left alone — it records what wasmeasured then.
Verification
Five mutants, all caught: reordering the sweep so it no longer ends on 0.5;
desynchronising the two languages' resolution lists; dropping a band; widening
one to vacuity; and reverting R to a single-resolution call. The band check was
separately confirmed to fire by narrowing one until it failed.
1179 passed, 25 skipped. mypy at its 4-error baseline.
--reportexits zero.A changed ridge-plot PNG was reverted as rendering churn, the underlying
clusters having been verified identical.
Also
Generalises the CSV-precision lint from #94:
dtype=strparses no floats, so itcannot misparse one. That was a per-file exemption for the DE hex reader and is
now a rule — which is what caught this PR's own new reads.
🤖 Generated with Claude Code