feat: migrate multimapper-tiebreak RNG to in-tree splitmix64 - #118
Closed
BenjaminDEMAILLE wants to merge 2 commits into
Closed
feat: migrate multimapper-tiebreak RNG to in-tree splitmix64#118BenjaminDEMAILLE wants to merge 2 commits into
BenjaminDEMAILLE wants to merge 2 commits into
Conversation
Replace the `rand` crate (StdRng) with STAR-rs's splitmix64-based deterministic generator for the two RNG consumers ported so far: `shuffle_tied_prefix` (SE/PE `--outMultimapperOrder Random` tie-break) and `pick_primary_and_mapq` (transcriptome-space primary pick). Per docs-old/dev/porting-from-star-rs.md's RNG decision: this must land before STARsolo, whose UMI dedup depends on deterministic read order. Keeps the existing per-read seed (run_rng_seed + read-name hash, already thread-invariant); only the underlying generator changes, matching STAR-rs's own choice of an in-tree splitmix64 over a general-purpose PRNG crate (documented in STAR-rs's DIVERGENCES.md D16: STAR's own mt19937 output is itself thread-count-dependent, so no external crate's output is any more "correct" to match). Also migrates the test-only packed_stream.rs fuzz generator off `rand`, so the dependency can be dropped from Cargo.toml entirely. Gate green: cargo build --all-targets, clippy -D warnings, fmt, 455 tests.
Collaborator
|
This is worth adopting early, as it should clean up the comparisons/tests quite a lot. We can later assess if we continue to use this method at a later date. |
Collaborator
|
If you keep all your PRs on the same base commit, I can take care of merging them all. So no need to update each one. |
Contributor
Author
Oh fine so. will let you do the merge :) |
Psy-Fer
added a commit
that referenced
this pull request
Jul 27, 2026
# Drop the `rand` dependency for an in-tree splitmix64 RNG (completes #118) Replaces the external `rand` crate with a small in-tree deterministic RNG (`src/rng.rs`, splitmix64). This is #118's original goal — done correctly this time, and only after **empirically proving the RNG choice is output-neutral** ## Why rustar-aligner's only randomness is tie-breaking and EmptyDrops Monte-Carlo — paths where the exact RNG stream is **not** STAR-faithful anyway (STAR uses mt19937; we already diverge and report a tie-adjusted metric). What actually matters is **determinism** (reproducible regardless of thread scheduling), which a fixed-seed splitmix64 gives without pulling in `rand` → `rand_core` → `rand_chacha` → `ppv-lite86` → `zerocopy`. Fewer deps = simpler build and a step toward the pure-Rust / C-toolchain-free goal. #118 (BenjaminDEMAILLE) proposed exactly this but just deleted `rand` from `Cargo.toml` without migrating the `WeightedIndex` users, so it broke the build. This PR does the migration. ## Empirical validation (the key point) Before touching anything, I measured whether the RNG choice changes results, on the mouse 10x set (`test/data/singlecell/`, `--soloCellFilter EmptyDrops_CR`) — the one path claimed to "need `rand`": - **StdRng vs splitmix64: identical 229 cells (0 differ).** The RNG choice makes no difference to the EmptyDrops calls. - **Both hit 228/229 vs STAR** — same single borderline FDR cell (Monte-Carlo noise), so splitmix64 is exactly as STAR-faithful as StdRng. - Deterministic across thread counts (4-thread ≡ 1-thread). So `WeightedIndex` is just a cumulative-weight sampler (~15 lines), and removing `rand` costs nothing in faithfulness. ## Changes - **`src/rng.rs`** (new): `SplitMix64` (`next_u64`/`uniform`/`below`), `shuffle_deterministic` (Fisher–Yates), `cumulative_weights` + `sample_cumulative` (`WeightedIndex`-equivalent). Unit-tested (determinism, range, permutation, weighted distribution). - Migrated all four production `rand` sites to it: - `solo/count.rs` — EmptyDrops ambient Monte-Carlo sampler. - `src/bin/emptydrops.rs` — standalone binary, same sampler. - `align/read_align.rs` — `--outMultimapperOrder Random` tie-break shuffle. - `lib.rs::pick_primary_and_mapq` — `--runRNGseed` primary tie-break. - (+ one `#[cfg(test)]` in `index/packed_stream.rs`.) - Dropped `rand` from `Cargo.toml`; `rand`/`rand_core`/`rand_chacha`/`ppv-lite86`/`zerocopy` are gone from the dependency tree. ## Validation - EmptyDrops: **229 cells, byte-identical** to the pre-change StdRng output; 228/229 vs STAR; deterministic across thread counts. - Yeast benchmark unchanged: **SE 8788/8926**, **PE 8390 both-mapped** (the tie-break stream change is output-neutral — tie-adjusted faithfulness holds). - **579 tests pass** (+5 new `rng` unit tests) · `clippy --all-targets` 0 warnings · `fmt` clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
randcrate (StdRng) with STAR-rs'ssplitmix64-based deterministic generator for the two RNG consumers:shuffle_tied_prefix(SE/PE--outMultimapperOrder Randomtie-break) andpick_primary_and_mapq(transcriptome-space primary pick).docs-old/dev/porting-from-star-rs.md's RNG decision: this must land before STARsolo, whose UMI dedup depends on deterministic read order.run_rng_seed+ read-name hash, already thread-invariant) — only the underlying generator changes, matching STAR-rs's own choice of an in-tree splitmix64 over a general-purpose PRNG crate (STAR-rs'sDIVERGENCES.mdD16: STAR's own mt19937 output is itself thread-count-dependent, so no external crate's output is any more "correct" to match).packed_stream.rsfuzz generator offrand, so the dependency drops fromCargo.tomlentirely.Test plan