Skip to content

fix(ledger): recover feedless-crypto cutover owed openings via a widened last-mark lookup#4275

Merged
TaprootFreak merged 2 commits into
developfrom
fix/ledger-cutover-feedless-owed
Jul 19, 2026
Merged

fix(ledger): recover feedless-crypto cutover owed openings via a widened last-mark lookup#4275
TaprootFreak merged 2 commits into
developfrom
fix/ledger-cutover-feedless-owed

Conversation

@TaprootFreak

Copy link
Copy Markdown
Collaborator

Closes #4270.

Problem

A cutover owed (openBuyCryptoOwed) or manual-debt (openManualDebt) opening whose asset carries no mark in the 2-day preload window — a delisted/feedless crypto — currently fails loud (bookReceivedOwedOpening throws), so the whole cutover rolls back and retries every 5 minutes forever. It is defensive today (0 such rows for the current snapshot) but a real wedge risk for a future delisting (e.g. the DeFiChain sunset).

The value is recoverable: the owed buy_crypto row is ≤ OPEN_ROW_LOOKBACK_DAYS (90d) old and an asset cannot be sold unpriced, so its last finite mark ≤ snapshot sits within a 90d window (365d for a standing manual-debt) — one the 2d preload and the 5d getLatestMark bridge simply don't reach.

Change

  • LedgerMarkService.getMarkAtWidened(assetId, asOf, lookbackDays) — read-only, lazy, memoized. Reuses the canonical preload → LedgerMarkCache.getMarkAt (binary-search "latest finite priceChf ≤ asOf") over a widened, daily-sampled window; the per-window cache is memoized so several feedless rows share one read. Returns undefined only for a truly-unpriced asset.
  • openBuyCryptoOwed / openManualDebt — try the 2d full-tick cache first, then getMarkAtWidened as a ?? fallback (90d / 365d). The opening is valued at amount × last-mark and booked as fixed CHF on the existing CHF bucket LIABILITY/{bucket} (assetId NULL, needsMark false) — byte-identical to a priced owed opening. A still-missing mark logs an alarm and fails loud (deferred), unchanged.

Settlement code (payout-order.consumer, bank-tx.consumer) is not touched.

Why this design (S) and not the issue's asset-backed sketch (A)

The issue proposed booking the opening asset-backed on a per-asset account and teaching the discharge to close it. A design review rejected that: to be mark-to-market-revaluable the per-asset account (LIABILITY/{bucket}/{asset}, amountChf null) re-introduces the exact name-mismatch discharge trap that closed #4251 (the forward cutoverOwedOpeningChf matches the CHF-bucket name + reads leg.amountChf), and mark-to-market can't revalue a never-relisted delisted asset anyway (its own 2d getMarkAt is undefined) — so A's revaluation benefit is vacuous while its hot-settlement-code surgery is real risk. Valuing on the existing CHF bucket keeps the delisted subset consistent with every other owed opening (all historical-cost CHF), needs zero settlement-code change, and closes cent-exact via the existing withFxPlug (opening↔settlement drift → INCOME/EXPENSE fx-revaluation). The final settled state is bit-for-bit identical to A; only the timing of fx recognition differs. The issue's own Notes already sanctioned "keep the opening on the CHF bucket".

Verification (Node 20)

prettier / eslint / type-check / build clean · DI graph resolves (boot reaches JwtStrategy env, i.e. past DI) · full accounting suite 776/776.

Added tests:

  • ledger-mark.service.specgetMarkAtWidened: recovers a mark older than the 2d/5d windows; reads over asOf − lookbackDays (dailySample); never returns a future mark; undefined for a truly-unpriced asset; memoizes per window (single read).
  • ledger-cutover.service.spec — a feedless buyCrypto-owed asset is recovered and booked on LIABILITY/buyCrypto-owed at amount × last-mark with no per-asset account (guards the fix(accounting): value foreign-fiat owed openings via the currency asset mark #4251 trap); a live asset is valued from the 2d cache without invoking the fallback; a feedless manual-debt is recovered over 365d; the truly-feedless cases still fail loud.

Coverage note: because a feedless opening is booked byte-identically to a priced owed opening (asserted directly by the cutover spec: same CHF-bucket account, real amountChf, needsMark false, no per-asset account), the forward discharge is unchanged code exercising an identical input — its cent-exact close is already covered by the existing priced-owed discharge tests. A separate payout_order integration harness was judged disproportionate for a defensive, 0-rows-today change; happy to add one if reviewers prefer.

…ned last-mark lookup

A cutover owed / manual-debt opening whose asset carries no mark in the 2-day preload
window (a delisted/feedless crypto) fails loud and wedges the whole cutover in its
5-minute retry loop. Yet the asset was necessarily priced when the <=90d-old owed row was
created, so its last finite mark <= snapshot sits within a 90d (buy_crypto) / 365d
(manual-debt) window the 2d preload and the 5d getLatestMark bridge do not reach.

Add LedgerMarkService.getMarkAtWidened(assetId, asOf, lookbackDays) - read-only, lazy,
memoized, reusing the canonical preload -> getMarkAt path - and use it as a `?? fallback`
in openBuyCryptoOwed / openManualDebt. The opening is then valued at outputAmount x
last-mark and booked as fixed CHF on the EXISTING CHF bucket (LIABILITY/{bucket}, assetId
NULL, needsMark false), byte-identical to a priced owed opening, so the unchanged
payout_order / bank_tx discharge closes it cent-exact to 0 - no per-asset account, no
settlement-code change. A truly-never-priced asset still yields undefined -> fail-loud
(deferred, alarmed).
The per-window preload promise was memoized before it resolved and never evicted on failure. A transient DB error on the widened read would cache a rejected promise under the pinned-snapshot key, so every 5-min cron retry re-threw without re-reading -> the cutover would stay wedged until a process restart even after the DB recovered. Evict the key on rejection so the next run re-reads.
@TaprootFreak

Copy link
Copy Markdown
Collaborator Author

Reviewed across two dimensions (conformance/quality + logic/correctness), two passes to zero findings.

  • Pass 1 raised one real minor: getMarkAtWidened memoized the per-window preload promise before it resolved and never evicted it on rejection, so a transient DB error on the widened read could cache a rejected promise under the pinned-snapshot key and re-throw on every 5-min retry — wedging the cutover until a process restart, even after the DB recovered. Fixed in 2a0e57b (evict the key on rejection) with a regression test.
  • Pass 2 clean (the only raised item was a refuted comment-wording nit — the 90d price-existence guarantee vs the 92d lookback is already explicit in the code, the log message, and the spec).

CI green (Build and test, review, CodeQL, analyze). Verified on Node 20: prettier / eslint / type-check / build, DI graph resolves, full accounting suite 776/776 (incl. the added cases).

@TaprootFreak
TaprootFreak marked this pull request as ready for review July 18, 2026 22:47
@TaprootFreak
TaprootFreak merged commit 8f50fae into develop Jul 19, 2026
7 checks passed
@TaprootFreak
TaprootFreak deleted the fix/ledger-cutover-feedless-owed branch July 19, 2026 07:42
TaprootFreak added a commit that referenced this pull request Jul 19, 2026
…ndow self-heals (#4281)

* fix(ledger): expire the widened last-mark memo so a still-feedless window self-heals

getMarkAtWidened (#4275) memoized its per-window preload on the process-lifetime
LedgerMarkService with no TTL — only a rejected load was evicted. A *successful*
preload whose immutable LedgerMarkCache lacks the asset (a still-feedless/delisted
mark) was therefore cached forever. Because the (from,to) key is byte-stable across
every 5-min cutover cron retry (the snapshot date is pinned), each retry re-read the
same stale-empty cache -> fail-loud -> the cutover wedged until a process restart,
defeating the "retry once the feed is back" recovery #4275 documents (asymmetry vs
the TTL'd latestMarks bridge).

Give widenedCaches the same self-healing TTL as latestMarks: a stale entry (incl. a
successful-but-empty one) expires after WIDENED_MARK_TTL_MS so a later retry re-reads
and can pick up a backfilled historical mark; within one run (seconds) the memo still
dedupes several feedless rows sharing the window. Liveness-only fix — no
financial-correctness impact (fail-loud stays safe).

Follows up #4275 (#4270).

* fix(ledger): address review nits on the widened-mark TTL

- guard the rejected-preload eviction by loadedAt so a late-rejecting stale load
  can never delete a fresher entry a later retry installed (a widened window now
  gets replaced after the TTL, so unconditional delete-by-key was newly unsafe);
  net effect was only an extra read, never a misbooking, but tighten it anyway.
- make the Date.now spy restore fail-safe via a describe-level afterEach so a
  failing assertion cannot leak the mock (the jest config has no restoreMocks).

No behaviour change on the happy/transient-failure/self-heal paths.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ledger cutover: value delisted/feedless-crypto owed openings asset-backed (with correct forward discharge)

1 participant