Read every tutorial CSV with float_precision="round_trip" - #94
Merged
Conversation
pandas' default CSV reader is not correctly rounded -- it misparses about a
third of random doubles by an ULP. `to_csv` was never the problem; it already
writes the shortest round-trippable form. So a tutorial that wrote a value,
handed it to R, read both back and reported "these agree exactly" was partly
measuring the parser rather than the port.
37 call sites across 16 tutorials fixed. Three reads stay exempt with stated
reasons: cell labels in anchors_tutorial, a row count wrapped in len() in
visium_tutorial, and the DE hex reader, which parses floats via float.fromhex.
Six reported figures moved, all at the ULP level and most downward: PBMC 8k's
percent.mt 5.773e-15 -> 5.329e-15, PBMC 3k's VST mean relative difference
1.548e-14 -> 4.973e-15 and its variance max|diff| 1.592e-11 -> 1.614e-11,
SCTransform's detection_rate 5.6e-16 -> 5.0e-16, CITE-seq's minimum 2.082e-16
-> 2.498e-16, and one PBMC 3k marker row 4.80e-14 -> 4.62e-14. No declared band
moved; all nine --report runs still exit zero, and re-running them reproduces
the new values exactly. The four affected vignettes carry the measured numbers.
The 0.9.0 changelog entries quoting the old figures are left alone -- they were
accurate at that release and a changelog is a record, not a live table.
The first attempt at this patch inserted the argument before the last ")" on
the line, which for `pd.read_csv(path).set_index("cell")` lands it inside
set_index. That is syntactically valid, passes an AST check, and would have
raised TypeError at runtime on 16 sites. Redone with paren matching from the
read_csv( position, and every changed line read back.
tests/test_tutorial_csv_precision.py pins the convention across all eighteen
tutorials so the next pd.read_csv cannot quietly reintroduce it. It also
asserts the three exemptions still match something -- a stale exemption stops
matching and silently permits a real offender in the same file. Both guards
mutation-tested.
Also corrects a bound the docs got wrong independently of this: PBMC 3k claimed
avg_log2FC agreement "to 4.9e-15" for both clusters whose cells match exactly.
That is the 151-gene cluster; the 242-gene one is 4.62e-14, ten times larger.
Both figures are now stated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #93, which fixed this in the DE tutorial only and left 42 read
sites across 17 others.
pandas' default CSV reader is not correctly rounded — it misparses about a
third of random doubles by an ULP.
to_csvwas never at fault; it alreadywrites the shortest round-trippable form. So a tutorial that wrote a value,
handed it to R, read both back and reported "these agree exactly" was partly
measuring the parser rather than the port.
What changed
37 call sites across 16 tutorials. Three reads stay exempt with stated
reasons: cell labels in
anchors_tutorial, a row count wrapped inlen()invisium_tutorial, and the DE hex reader, which parses floats viafloat.fromhex.Six reported figures moved, all at the ULP level and most of them downward:
percent.mtdetection_rateNo declared band moved. All nine
--reportruns still exit zero, andre-running them reproduces the new values exactly. The four affected vignettes
carry the measured numbers. The
[0.9.0]changelog entries quoting the oldfigures are left alone — they were accurate at that release, and a changelog is
a record rather than a live table.
A near miss worth recording
The first attempt inserted the argument before the last
)on the line. Forpd.read_csv(path).set_index("cell")that lands it insideset_index—syntactically valid, passes an AST check, and would have raised
TypeErroratruntime on 16 of the 37 sites. Caught only by reading back every changed line
rather than trusting the parse. Redone with paren matching from the
read_csv(position.Guard
tests/test_tutorial_csv_precision.pypins the convention across all eighteentutorials so the next
pd.read_csvcannot quietly reintroduce it. It alsoasserts the three exemptions still match something — a stale exemption stops
matching and silently permits a real offender in the same file. Both guards
mutation-tested: stripping the parameter from one call fails that file's case,
and editing an exempt call's marker fails both the file case and the
exemption check.
Also corrected
PBMC 3k's docs claimed
avg_log2FCagreement "to 4.9e-15" for both clusterswhose cells match exactly. That covers the 151-gene cluster (4.88e-15); the
242-gene one is 4.62e-14, ten times larger. Both figures are now stated.
This predates the reader fix — the bound was simply the smaller of the two.
Verification
1168 passed, 25 skipped. mypy at its 4-error baseline. No figure churn.
Not in this PR
The R half is not lintable from Python and is unchanged:
write.csvrenders 15significant digits, and raising it does not help because R's own
sprintf("%.17g")is not correctly rounded either. Only the DE tutorial has theC99 hex-float (
%a) side table. The tutorials making the strongest exactnessclaims — the object model ("91 of 91 anchors, no tolerance"), Visium
(
max|Δx| = 0), out-of-core ("bit-identical"), and spatial (Moran's I to1.6e-14) — still cross the language boundary through plain CSVs.
🤖 Generated with Claude Code