solo: MultiGeneUMI_All (closes #144), and bit-exact libc++ RNG for EmptyDrops - #152
Open
BenjaminDEMAILLE wants to merge 2 commits into
Open
solo: MultiGeneUMI_All (closes #144), and bit-exact libc++ RNG for EmptyDrops#152BenjaminDEMAILLE wants to merge 2 commits into
BenjaminDEMAILLE wants to merge 2 commits into
Conversation
…GeneUMI `--soloUMIfiltering MultiGeneUMI_All` resolved to the same variant as `MultiGeneUMI`, which is neither what STAR does nor what the option is documented to do. Of the three available behaviours it was the only one nobody had asked for. In STAR the option is a no-op: it is parsed and stored, but its consumption site tests only the `MultiGeneUMI` flag, so selecting it leaves the filter entirely off. Documented, it removes a UMI seen in more than one gene from *all* of them, rather than from the losers only. `UmiFiltering::MultiGeneUmiAll` now exists and does the documented thing: a UMI appearing in several genes is evidence of a collision or of chimeric amplification, so it is discarded outright rather than attributed to whichever gene happened to read deepest. Single-gene UMIs are untouched, which the test checks across every mode. Raised upstream as scverse#144 before changing it, since "be faithful to STAR" and "do what the flag says" genuinely point in opposite directions here. Also adds `docs-old/dev/divergences.md`, recording this and the homopolymer-UMI rule, so deliberate differences are written down rather than rediscovered as surprises in a differential run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…_distribution STARsolo's `EmptyDrops_CR` rescue draws from `std::mt19937`, converts to doubles with `std::generate_canonical<double, 53>`, and picks categories with `std::discrete_distribution`. Two of those three are implementation-defined in the parts that matter: the standard fixes mt19937's output but not how `generate_canonical` consumes it, and says nothing about how `discrete_distribution` maps a uniform onto categories. So porting "the algorithm" is not enough — it has to be libc++'s algorithm, because that is what STAR is built against and where its numbers come from. libc++ accumulates two 32-bit draws in *ascending* significance and divides by 2^64; a most-significant-first accumulation, or one draw scaled to 53 bits, both give perfectly good uniforms and neither reproduces STAR. Every expected value in the tests came out of a C++ program compiled against the real libc++ and run, not from reading its source. `tests/libcxx_oracle.cpp` is that program, kept so the values can be regenerated rather than trusted. `generate_canonical` is compared as bit patterns, since a difference in the last place changes which category a sample lands in. Not yet wired into the EmptyDrops path. `solo::count` samples with a `SplitMix64` stream under a comment calling it "WeightedIndex-equivalent; empirically byte-identical EmptyDrops cell calls" — a claim that cannot hold in general, since two unrelated generators cannot agree on an arbitrary number of draws. It is true of whatever was checked and unknown elsewhere. Replacing it moves cell calls, so it belongs in its own change with the solo differential run against it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE
changed the title
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
--soloUMIfiltering MultiGeneUMI_All(closes #144), and a bit-exact libc++ RNG for EmptyDrops.What changed
MultiGeneUMI_Alldoes what it is documented to do: a UMI seen in more than one gene is removed from all of them.MultiGeneUMIflag, so selecting it leaves the filter offMultiGeneUMI: keep the deepest-read geneSingle-gene UMIs are untouched, asserted across every mode.
src/solo/libcxx_rng.rsports the three libc++ facilities STARsolo'sEmptyDrops_CRdraws from:std::mt19937,std::generate_canonical<double, 53>andstd::discrete_distribution. The standard fixes mt19937's output but not howgenerate_canonicalconsumes it, nor howdiscrete_distributionmaps a uniform draw onto categories, so reproducing STAR's numbers requires libc++'s algorithm specifically.Why
Reproducing the no-op ships a flag that silently does nothing to anyone who read STAR's documentation. #144 records the divergence; the choice was made there before any code changed. The test asserts the behaviour either way, so inverting it is a one-line change if that is preferred.
The RNG port exists because
solo::countsamples with aSplitMix64stream under a comment calling it "WeightedIndex-equivalent; empirically byte-identical EmptyDrops cell calls". That claim cannot hold in general: two unrelated generators cannot agree over an arbitrary number of draws. It is true of whatever cases were checked and unknown elsewhere.Verification
Every expected value in the RNG tests was produced by compiling a C++ program against real libc++ (
clang++ -stdlib=libc++) and printing the results, not derived from reading its source.tests/libcxx_oracle.cppis kept in-tree so the values can be regenerated.generate_canonicalis asserted on bit patterns, not decimals: the point is that the double matches to the last place, since a difference there changes which category a sample lands in.561 lib + 26 integration green, clippy
-D warningsclean,cargo fmt --checkclean, MSRV 1.89.The RNG is not yet wired into the EmptyDrops path. Replacing the sampler moves cell calls, which wants its own change with a solo differential behind it rather than a quiet swap inside a UMI-filtering PR.
This is a counting change on the
MultiGeneUMI_Allpath only; the default (-) and the other modes produce identical output.Also
Adds
docs-old/dev/divergences.md. Two entries: this one, and the homopolymer-UMI rule from #150 where STAR'sumiL = 0quirk lets poly-C, poly-G and poly-T through its filter. The file also records one divergence deliberately not taken: STAR-rs resolves the transcriptome-BAM primary flag by always taking the first alignment, while this codebase picks by a per-read seed, which is thread-count invariant and fixes the same defect.🤖 Generated with Claude Code