aligner core: annotated-junction stitching, alignEndsType, in-recursion length penalty - #146
Open
BenjaminDEMAILLE wants to merge 8 commits into
Open
Conversation
…ind() STAR's stitcher resolves an annotated junction by index (`sjA` / `sjdbInd`) into `mapGen.sjdb*`, then reads that entry's motif, shiftLeft and shiftRight. rustar-aligner had only a boolean `is_annotated()` over a HashMap, so neither the `sjAB` fast path nor annotated-boundary snapping could be expressed. Add the missing half: - `SpliceJunctionDb::table: Vec<PreparedJunction>`, the annotated junctions in the exact order they occupy the Gsj buffer. - `find(stored_start, stored_end) -> Option<usize>`, a port of STAR's `binarySearch2`: binary search on the donor, then a backward/forward scan of the equal-donor run for the acceptor. Coordinates are genome-absolute, so no chromosome index is needed. - `entry(i)`, `table_len()`, `from_prepared()`, `set_table()`. No index format change. `PreparedJunction` already carries motif, shift_left, shift_right and strand; `sjdbInfo.txt` already writes and reads all four; and `sort_and_dedup` already establishes the `(stored_start, stored_end)` ordering the search needs. Existing indices work unchanged. The three population paths now agree: - `genomeGenerate` builds the db from the same `prepared` array it feeds to `build_gsj`, so table order and Gsj order match by construction. - The index-load path uses `from_prepared(prepared_junctions)`. - The align-time `--sjdbGTFfile` path derives motif/shift via `prepare_junction` + `sort_and_dedup`, which it previously discarded. Whenever the index carries junctions, its array wins as the table even if a GTF is also supplied at align time: `sj_a` tags come from `decode_gsj_hit`, which indexes that array, so anything else would make a tag address the wrong junction. Behaviour-neutral for canonical junctions, where stored coordinates equal the raw GTF ones. For non-canonical junctions the `genomeGenerate` in-memory db is now keyed on stored coordinates like the other two paths, which is what the stitch-time scan produces. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ted-junction shortcut STAR tags every seed derived from the sjdb genome insert with the index of the junction it came from (`sjA`), then, when the previous exon and the incoming piece carry the *same* tag and are read-adjacent, stitches them straight onto the annotated junction instead of searching for a boundary. rustar-aligner computed that index and threw it away. Plumbing: - `decode_gsj_hit` returns the `junction_idx` it already computed. Both halves of a boundary-crossing hit carry it: they are two flanks of one junction. - `expand_hit`, `WinCandidate`, `WindowAlignment` and `ExonBlock` carry `sj_a`, `-1` for ordinary real-genome hits. - The overlap dedup in `cluster_seeds` now compares `sj_a` alongside `mate_id`, matching STAR's `e.sjA == sjA` guard in `assignAlignToWindow`. Seeds derived from different annotated junctions are no longer collapsed just because they share a diagonal. The shortcut itself, at the head of `stitch_align_to_transcript`: - fires when both sides carry the same non-`-1` tag, sit on the same mate, are exactly read-adjacent, and are separated on the genome; - takes exon B verbatim at the annotated boundary, adopts the stored motif and shifts, marks the junction annotated, and adds `sjdbScore`; - rejects (STAR's -1000006, `None` here) when micro-repeats around a non-canonical annotated junction are larger than the pieces being stitched. Without it, the general boundary search flushes a repeat-adjacent junction to its leftmost position and lands a few bases off the annotated coordinates (STAR's `11M1545N38M` becoming `9M1545N40M`), which also forfeits the annotated bonus because the shifted coordinates no longer match the database. Receiver coordinates are half-open where STAR's are inclusive, so `rBstart == rAend + 1` is `wa.read_pos == last_exon.read_end` and `gAend + 1 < gBstart` is `last_exon.genome_end < wa.sa_pos`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ted coordinates Closes item #4b of STAR-RS-COMPARISON.md §7.2, which §7.4 calls "the best remaining lever". The splice branch previously asked the annotation a yes/no question and, on a hit, swapped the motif penalty for `sjdbScore`. It never used what the annotation actually says about *where* the junction is. STAR does three things with the entry it finds: 1. adopts the stored motif (`jcan = sjdbMotif[ind]`), so the mismatch gate, `outFilterIntronMotifs` and SJ.out.tab all see the annotated motif rather than one re-derived from the genome; 2. for a non-canonical entry, shifts `jR` by `shiftLeft`, moving the boundary off the leftmost-flush position the scan picked and onto the annotated one; 3. rejects (-1000006) when the micro-repeat is larger than either piece, or when the shift would push the donor past the end of piece B. All three are now done. Because `jr_shift` is consumed afterwards to move exon A's right edge and exon B's left edge together, shifting it relocates the whole junction coherently and leaves the intron length untouched. The mismatch gate moves after the lookup, matching STAR's ordering: it must test the annotated motif, not the scanned one. This changes behaviour on annotated non-canonical junctions, which is the point. `find()` and `is_annotated()` are queried separately rather than one wrapping the other: junctions inserted by two-pass mode live only in the map and carry no motif or shift, so they must still count as annotated while contributing nothing to snap. New tests: `noncanonical_annotated_junction_snaps_to_annotated_coords` measures where the unannotated search lands, annotates exactly that junction, and asserts both boundaries move by `shiftLeft` while the intron keeps its length (and that a canonical entry at the same coordinates is left alone). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ead-length mismatch cap Adds the flags this theme needs that scverse#145 did not bring: `alignEndsProtrude`, `alignSoftClipAtReferenceEnds`, `alignInsertionFlush`, `alignTranscriptsPerReadNmax`, `outFilterMismatchNoverReadLmax`, `seedNoneLociPerWindow`, `seedSplitMin`. Off-menu values are rejected loudly rather than silently ignored. `--alignEndsType` and its `ext[mate][end]` matrix come from scverse#145; this branch builds on that rather than duplicating it. The scorer gains only the three fields scverse#145 does not have: `flush_right`, `soft_clip_at_reference_ends` and `p_mm_max_read`. Two behaviours land here: - `--alignInsertionFlush Right` now works. The insertion-placement scan accepts ties as well as strict improvements, then walks the insertion further right for as long as the read keeps matching, rejecting when it runs out of read on the B side (STAR -1000009). `None`, the default, is bit-identical to before. - `--outFilterMismatchNoverReadLmax` is enforced. STAR's `outFilterMismatchNmaxTotal` is the tightest of three caps: absolute, a fraction of the mapped length, and a fraction of the read length. Only the first two were applied, so the flag had nothing to bind on. Now in `AlignmentScorer::mismatch_nmax_total`. Note for a follow-up: the pre-existing `--clipAdapterType` validation sits inside the `solo_enabled()` branch of `try_parse_from`, so it only fires for STARsolo runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An alignment may no longer be soft-clipped past the end of its chromosome when `--alignSoftClipAtReferenceEnds No` is given. `Yes`, the default, leaves behaviour unchanged. Sits directly after the end-to-end boundary check from scverse#145: both are "this extension may not run off the reference" rules and share the extension results. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ot after it Closes item scverse#7 of STAR-RS-COMPARISON.md §7.2. STAR applies `scoreGenomicLengthLog2scale` in the base case of `stitchWindowAligns`, before the dedup and eviction that decide which transcripts survive the window. rustar-aligner applied it only in `finalize_transcript`, which runs after those decisions, so the recursion ranked candidates on unpenalised scores: a sprawling alignment could displace a compact one of equal raw score, and the penalty arrived too late to matter. `WorkingTranscript` gains `rank_score` = `score` + genomic-length penalty, computed in the base case and used by the three places that compare siblings: the two `same_structure` dominance tests and the `min_by_key` eviction. `rank_score` is kept separate from `score` rather than folded into it. The PE mate-split path derives two per-mate transcripts from one combined score, and splitting an already-penalised score across mates has no clean semantics. `finalize_transcript` still applies the penalty to the final score exactly as before, so nothing is double-counted. This changes which alignment wins among equal-raw-score candidates, which is the point. §7.5's lesson from the dropped item scverse#2 applies: if the differential measures negative, this commit is the one to revert. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tsPerReadNmax
Adds `test/simulate_junction_reads.py` and a `yeast_junction_gtf` tier.
The existing tiers align real RNA-seq reads, of which only a small fraction
cross an intron — yeast has a few hundred — so the annotated-junction code
paths are barely exercised and a divergence in them is easy to miss. The
simulator draws reads from spliced transcript sequences centred on annotated
junctions, half of them reverse-complemented, from a fixed seed so the FASTQ is
reproducible. Every read crosses an annotated junction.
Also wires `--alignTranscriptsPerReadNmax`: STAR breaks out of the window loop
once one more window's worth of transcripts could overrun the per-read cap, so
the cap is never exceeded rather than merely noticed afterwards.
Measured with this tier, 1000 reads, yeast R64-1-1 with the Ensembl 110 GTF,
against native STAR 2.7.11b, records compared on FLAG/RNAME/POS/CIGAR:
origin/main 835 / 1000 identical
this branch 933 / 1000 identical
gained 98, regressed 0
junctions (total / annotated)
STAR 155 / 155
origin/main 149 / 131
this branch 149 / 131
The gain is entirely in CIGARs: the junction set is unchanged, but 98 reads now
place their exon boundaries where STAR places them. Against an unannotated
index both binaries score 938/1000, gained 0, lost 0, confirming the
in-recursion length penalty is neutral there and the whole improvement comes
from annotated-boundary snapping.
Output is byte-identical at `--runThreadN` 1, 4 and 16.
Adds `docs-old/dev/divergences.md`, recording the two places this branch
knowingly differs from STAR: the extend-to-end rejection signal, and the
missing per-junction strand on non-canonical annotated junctions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scverse#145 handles the same case with an explicit EXTEND_TO_END_KILL sentinel checked before the score clamp, so there is no divergence left to record. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BenjaminDEMAILLE
force-pushed
the
bd/aligner-core-faithfulness
branch
from
July 28, 2026 20:21
a35f265 to
3661533
Compare
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Aligner-core faithfulness: annotated-junction stitching,
alignEndsType, and the genomic-length penalty inside the stitch recursion.What changed
sjAtags survive the seed pipeline.decode_gsj_hitcomputed the junction index and discarded it; it is now carried throughexpand_hit→WinCandidate→WindowAlignment→ExonBlock. The overlap dedup incluster_seedsgained STAR'se.sjA == sjAguard, so seeds from different annotated junctions are no longer collapsed for sharing a diagonal.The annotated shortcut. When the previous exon and the incoming piece carry the same tag, sit on the same mate and are read-adjacent, they are the two flanks of one junction that the Gsj insert split apart. They are stitched directly onto the annotated boundary with the stored motif, shifts and
sjdbScorebonus, skipping the general search.Annotated-boundary snapping. A non-canonical annotated junction is shifted by
shiftLeftonto its annotated coordinates instead of being left leftmost-flush, and the stored motif overrides the one re-derived from the genome. Both of STAR's-1000006rejections are reproduced.The genomic-length penalty moves inside the recursion, where STAR applies it, rather than being added in a post-pass.
--alignEndsTypeLocal,EndToEnd,Extend5pOfRead1,Extend5pOfReads12)--alignSoftClipAtReferenceEnds--alignInsertionFlushRight;Noneis bit-identical to before)--outFilterMismatchNoverReadLmax--alignTranscriptsPerReadNmax--alignEndsProtrude,--seedNoneLociPerWindow,--seedSplitMinWhy
STAR-RS-COMPARISON.md§7.2 lists these as the open faithfulness items and §7.4 names annotated-boundary snapping the best remaining lever.stitch.rsasked the annotation a yes/no question and, on a hit, swapped the motif penalty forsjdbScore; it never used what the annotation says about where the junction is. Item #2 stays dropped, as §7.5 records it regressing.No index format change
PreparedJunctionalready carriedmotif,shift_left,shift_rightandstrand;sjdbInfo.txtalready wrote and read all four;sort_and_dedupalready established the(stored_start, stored_end)orderingbinarySearch2needs.find()searches data that was already on disk. Existing indices work unmodified.One inconsistency is fixed along the way: the align-time
--sjdbGTFfilepath went throughfrom_raw_junctions, which discarded motif and shift, while the index-loaded path kept them. The three population paths now build identical tables. Where the index carries junctions, its array is thesjAtable, because that is the arraydecode_gsj_hitindexes.Verification
Existing tiers align real RNA-seq reads, of which few cross an intron, so annotated-junction paths were barely exercised. This PR adds
test/simulate_junction_reads.pyand ayeast_junction_gtftier: 1000 reads drawn from spliced transcript sequences, centred on annotated junctions, half reverse-complemented, fixed seed.Against native STAR 2.7.11b on that tier, raw exact counts:
98 reads gained, 0 regressed.
Gate:
cargo build --release && cargo test --release,cargo clippy --all-targets -- -D warnings,cargo fmt --check, MSRV 1.89 — all green. Thread invariance checked at--runThreadN1/4/16.🤖 Generated with Claude Code