fix(naming): stop the de-novo loader forking the identity of already-…#21
Merged
Conversation
…named markers
The Variant Naming Authority queue was ~86% noise: 130,553 of 151,318 entries
were branch-defining variants whose marker YBrowse had already named. Clicking
one showed "a named variant already exists at this locus" — and offered no
button that could reuse it. The only actions were Mint DU name, which would
give an already-named marker a second conflicting identity, and Flag for review.
Root cause is job ordering, not a missing check. resolve_variant /
prefill_vcache do match the catalog before minting, but only on
`coordinates @> {'hs1': ...}`, and the Y catalog's hs1 coords are populated by
a *separate later* job. The cutover ran ybrowse-ingest -> de-novo load ->
variant-coord-lift, so at load time FT186008 had only GRCh37/38 coords, the hs1
match found nothing, and the loader minted chrY:10249542C>G alongside it. The
duplicate is what gets linked to the branch, stranding the real name on an
unlinked catalog row.
- denovo: a Y load now refuses to start while any named Y catalog row lacks an
hs1 coordinate. The old behaviour degraded silently into mass duplication.
- naming: dedup_by_site excludes coordinate placeholders (it could report
"a named variant already exists: chrY:10249542C>G"), and its ordering is now
the adoption preference. adoptable_name() reads the *site twin*, not just the
row's own aliases — a de-novo row has no aliases, which is precisely why the
Reuse button never appeared for the rows the warning fired on.
- jobs: run-once variant-name-reconcile folds placeholder rows onto the name
they already have, using the same match and preference order as the button.
Idempotent, id-cursor batched. Wired nightly between resync-ybrowse and
name-private-nodes: a refresh that names a marker after a tree load recreates
this state, and node naming can only see a name once the variant carries it.
- mig 0068: indexes for the site lookup, previously a full scan of 3.1M rows on
every panel click.
Scoped to branch-defining rows. The loader also leaves ~94.6k branch-less
coordinate rows (~67k duplicating a named catalog row); they are linked to
nothing, never reach the queue, and cannot be renamed (they would collide on
(name, NULL)) — that needs a row-merge pass and is deliberately out of scope.
Verified on decodingus_cutover: 157,056 scanned / 130,553 adopted / 0 conflicts;
queue 151,318 -> 20,765; re-run adopts 0.
Production deploy is NOT done — see rust/docs/variant-name-reconcile-deploy.md.
core.variant is 4.7 GB and 0068 uses plain CREATE INDEX (SHARE lock, blocks
writes) while du-web migrates on boot; build the indexes CONCURRENTLY first so
the migration no-ops.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…named markers
The Variant Naming Authority queue was ~86% noise: 130,553 of 151,318 entries were branch-defining variants whose marker YBrowse had already named. Clicking one showed "a named variant already exists at this locus" — and offered no button that could reuse it. The only actions were Mint DU name, which would give an already-named marker a second conflicting identity, and Flag for review.
Root cause is job ordering, not a missing check. resolve_variant / prefill_vcache do match the catalog before minting, but only on
coordinates @> {'hs1': ...}, and the Y catalog's hs1 coords are populated by a separate later job. The cutover ran ybrowse-ingest -> de-novo load -> variant-coord-lift, so at load time FT186008 had only GRCh37/38 coords, the hs1 match found nothing, and the loader minted chrY:10249542C>G alongside it. The duplicate is what gets linked to the branch, stranding the real name on an unlinked catalog row.Scoped to branch-defining rows. The loader also leaves ~94.6k branch-less coordinate rows (~67k duplicating a named catalog row); they are linked to nothing, never reach the queue, and cannot be renamed (they would collide on (name, NULL)) — that needs a row-merge pass and is deliberately out of scope.
Verified on decodingus_cutover: 157,056 scanned / 130,553 adopted / 0 conflicts; queue 151,318 -> 20,765; re-run adopts 0.
Production deploy is NOT done — see rust/docs/variant-name-reconcile-deploy.md. core.variant is 4.7 GB and 0068 uses plain CREATE INDEX (SHARE lock, blocks writes) while du-web migrates on boot; build the indexes CONCURRENTLY first so the migration no-ops.