Skip to content

ledger: an unconfigured git backend makes every hourly tick re-anchor a quiet tip to Rekor #9646

Description

@JSONbored

⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.

Context

runScheduledLedgerAnchor (src/review/ledger-anchor-scheduler.ts:92) decides whether a tick should anchor,
then submits to Rekor and — only when the caller supplied one — to git.

Two lines are in conflict:

  • Line 101 asks which backends still lack a successful anchor for the current tip, always over the fixed list
    ANCHOR_BACKENDS_REQUIRING_SUCCESS = ["rekor", "git"] (line 90).
  • Line 142: if (deps.submitGit) { attempts.push(guardedSubmit("git", ...)) } — git is submitted only when the
    caller wired it.

The real caller, src/queue/job-dispatch.ts:104-114, builds submitGit as null unless
LOOPOVER_LEDGER_ANCHOR_GIT_OWNER, ..._REPO and ..._INSTALLATION_ID are all set. None of those is set in
wrangler.jsonc, config/, docs/, or .github/ anywhere in this repo — so submitGit is null in every
deployment today, exactly as job-dispatch.ts's own comment describes ("unset means that backend simply isn't
wired up yet").

Consequence: git never produces a status='ok' row, so anchorBackendsMissingForRowHash
(src/review/ledger-anchor-persistence.ts:182) returns ["git"] forever. In
decideLedgerAnchorSchedule (line 48) that makes the retry_unanchored arm true on every hourly tick, even
when the ledger is completely idle and the tip is already successfully anchored on Rekor:

if (input.isHourly && tipUnchanged && input.unanchoredBackends && input.unanchoredBackends.length > 0) {
  return { shouldAnchor: true, reason: "retry_unanchored" };
}

So the { shouldAnchor: false, reason: "unchanged" } quiet state is structurally unreachable on an hourly tick,
and the Worker submits a fresh, byte-identical hashedrekord entry to Rekor once an hour, indefinitely, for a
ledger where nothing has happened. That directly contradicts this module's own opening rationale ("checkpoint
cadence, not per-record … Rekor is a donated public good") and #9489's stated intent, which was to retry a
FAILED anchor, not to re-anchor a tip that already succeeded.

test/unit/ledger-anchor-scheduler.test.ts:235 pins the invariant "a fully anchored unchanged tip stays quiet"
— but only against decideLedgerAnchorSchedule called directly with unanchoredBackends: [], a value the real
runScheduledLedgerAnchor cannot produce while git is unconfigured. The pure function is right; the wiring is
wrong.

Requirements

  • runScheduledLedgerAnchor must compute the required-success backend list from the backends it will ACTUALLY
    attempt on this tick, not from a fixed constant: rekor always, plus git only when deps.submitGit is
    non-null.
  • The list computed for this tick must be the same list used both for anchorBackendsMissingForRowHash and for
    the submissions below it, so the two can never disagree again.
  • ots and bittensor must remain excluded from the required-success set, for the reason already documented at
    line 88-89 (tracked-but-not-built / operator-side submitter).
  • No change to decideLedgerAnchorSchedule's signature or semantics — the defect is entirely in what
    runScheduledLedgerAnchor feeds it.

⚠️ Required pattern: mirror the existing conditional at src/review/ledger-anchor-scheduler.ts:142
(if (deps.submitGit)) — the same deps.submitGit truthiness that gates the attempt must gate membership in
the required-success list. What does NOT satisfy this issue: deleting "git" from
ANCHOR_BACKENDS_REQUIRING_SUCCESS outright (that permanently blinds the retry for operators who DO configure
git anchoring); adding an env-var read inside ledger-anchor-scheduler.ts (resolveGitAnchorTarget is
deliberately the caller's job — job-dispatch.ts also needs the installation id, which this module never
sees); or suppressing the extra submissions with a time-based throttle instead of fixing the backend list.

Deliverables

  • runScheduledLedgerAnchor in src/review/ledger-anchor-scheduler.ts derives the backend list it passes
    to anchorBackendsMissingForRowHash from deps.submitGit, so git is required only when it will be
    attempted.
  • New named regression test in test/unit/ledger-anchor-scheduler.test.ts asserting: with submitGit
    omitted, a tip that already has a status='ok' rekor anchor row for its exact row_hash, and
    isHourly: true, runScheduledLedgerAnchor resolves to { shouldAnchor: false, reason: "unchanged" }
    and the injected submitRekor spy is called zero times.
  • Companion test asserting the retry still works when it should: same setup but with the rekor row recorded
    as status='failed'reason: "retry_unanchored" and submitRekor called once.
  • Companion test asserting git is still required when wired: submitGit supplied, rekor OK for the tip but
    git having no OK row ⇒ reason: "retry_unanchored", and both submitters invoked.

All Deliverables above are required in a single PR. A PR that satisfies only some of them — for example fixing
the backend list without the zero-call assertion in Deliverable 2, so a future regression re-opens the loop
silently — does not resolve this issue.

Test Coverage Requirements

This repo enforces 99%+ Codecov patch coverage, branch-counted, on src/**, and
src/review/ledger-anchor-scheduler.ts is inside coverage.include. Both arms of the new
deps.submitGit ? … : … conditional need a test (git wired / git not wired). The Deliverable-2 test is the
required named regression test for this fix.

Expected Outcome

On a deployment with git anchoring unconfigured (every deployment today), an hourly tick over an idle,
already-Rekor-anchored ledger performs no submission at all and reports reason: "unchanged", instead of
publishing a duplicate transparency-log entry every hour forever. Retry-on-failure and the per-backend
masking fix from #9489 both keep working unchanged.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions