Skip to content

fix(naming): stop the de-novo loader forking the identity of already-…#21

Merged
JamesKane merged 1 commit into
mainfrom
fix/variant-name-reconcile
Jul 13, 2026
Merged

fix(naming): stop the de-novo loader forking the identity of already-…#21
JamesKane merged 1 commit into
mainfrom
fix/variant-name-reconcile

Conversation

@JamesKane

Copy link
Copy Markdown
Owner

…named markers

The Variant Naming Authority queue was ~86% noise: 130,553 of 151,318 entries were branch-defining variants whose marker YBrowse had already named. Clicking one showed "a named variant already exists at this locus" — and offered no button that could reuse it. The only actions were Mint DU name, which would give an already-named marker a second conflicting identity, and Flag for review.

Root cause is job ordering, not a missing check. resolve_variant / prefill_vcache do match the catalog before minting, but only on coordinates @> {'hs1': ...}, and the Y catalog's hs1 coords are populated by a separate later job. The cutover ran ybrowse-ingest -> de-novo load -> variant-coord-lift, so at load time FT186008 had only GRCh37/38 coords, the hs1 match found nothing, and the loader minted chrY:10249542C>G alongside it. The duplicate is what gets linked to the branch, stranding the real name on an unlinked catalog row.

  • denovo: a Y load now refuses to start while any named Y catalog row lacks an hs1 coordinate. The old behaviour degraded silently into mass duplication.
  • naming: dedup_by_site excludes coordinate placeholders (it could report "a named variant already exists: chrY:10249542C>G"), and its ordering is now the adoption preference. adoptable_name() reads the site twin, not just the row's own aliases — a de-novo row has no aliases, which is precisely why the Reuse button never appeared for the rows the warning fired on.
  • jobs: run-once variant-name-reconcile folds placeholder rows onto the name they already have, using the same match and preference order as the button. Idempotent, id-cursor batched. Wired nightly between resync-ybrowse and name-private-nodes: a refresh that names a marker after a tree load recreates this state, and node naming can only see a name once the variant carries it.
  • mig 0068: indexes for the site lookup, previously a full scan of 3.1M rows on every panel click.

Scoped to branch-defining rows. The loader also leaves ~94.6k branch-less coordinate rows (~67k duplicating a named catalog row); they are linked to nothing, never reach the queue, and cannot be renamed (they would collide on (name, NULL)) — that needs a row-merge pass and is deliberately out of scope.

Verified on decodingus_cutover: 157,056 scanned / 130,553 adopted / 0 conflicts; queue 151,318 -> 20,765; re-run adopts 0.

Production deploy is NOT done — see rust/docs/variant-name-reconcile-deploy.md. core.variant is 4.7 GB and 0068 uses plain CREATE INDEX (SHARE lock, blocks writes) while du-web migrates on boot; build the indexes CONCURRENTLY first so the migration no-ops.

…named markers

The Variant Naming Authority queue was ~86% noise: 130,553 of 151,318 entries
were branch-defining variants whose marker YBrowse had already named. Clicking
one showed "a named variant already exists at this locus" — and offered no
button that could reuse it. The only actions were Mint DU name, which would
give an already-named marker a second conflicting identity, and Flag for review.

Root cause is job ordering, not a missing check. resolve_variant /
prefill_vcache do match the catalog before minting, but only on
`coordinates @> {'hs1': ...}`, and the Y catalog's hs1 coords are populated by
a *separate later* job. The cutover ran ybrowse-ingest -> de-novo load ->
variant-coord-lift, so at load time FT186008 had only GRCh37/38 coords, the hs1
match found nothing, and the loader minted chrY:10249542C>G alongside it. The
duplicate is what gets linked to the branch, stranding the real name on an
unlinked catalog row.

- denovo: a Y load now refuses to start while any named Y catalog row lacks an
  hs1 coordinate. The old behaviour degraded silently into mass duplication.
- naming: dedup_by_site excludes coordinate placeholders (it could report
  "a named variant already exists: chrY:10249542C>G"), and its ordering is now
  the adoption preference. adoptable_name() reads the *site twin*, not just the
  row's own aliases — a de-novo row has no aliases, which is precisely why the
  Reuse button never appeared for the rows the warning fired on.
- jobs: run-once variant-name-reconcile folds placeholder rows onto the name
  they already have, using the same match and preference order as the button.
  Idempotent, id-cursor batched. Wired nightly between resync-ybrowse and
  name-private-nodes: a refresh that names a marker after a tree load recreates
  this state, and node naming can only see a name once the variant carries it.
- mig 0068: indexes for the site lookup, previously a full scan of 3.1M rows on
  every panel click.

Scoped to branch-defining rows. The loader also leaves ~94.6k branch-less
coordinate rows (~67k duplicating a named catalog row); they are linked to
nothing, never reach the queue, and cannot be renamed (they would collide on
(name, NULL)) — that needs a row-merge pass and is deliberately out of scope.

Verified on decodingus_cutover: 157,056 scanned / 130,553 adopted / 0 conflicts;
queue 151,318 -> 20,765; re-run adopts 0.

Production deploy is NOT done — see rust/docs/variant-name-reconcile-deploy.md.
core.variant is 4.7 GB and 0068 uses plain CREATE INDEX (SHARE lock, blocks
writes) while du-web migrates on boot; build the indexes CONCURRENTLY first so
the migration no-ops.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JamesKane JamesKane merged commit c0592ea into main Jul 13, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant