Skip to content

fix(log): mark implausibly high FinancialDataLog entries as invalid#3884

Merged
TaprootFreak merged 3 commits into
developfrom
fix/finance-log-valid-flag-correction
Jun 15, 2026
Merged

fix(log): mark implausibly high FinancialDataLog entries as invalid#3884
TaprootFreak merged 3 commits into
developfrom
fix/finance-log-valid-flag-correction

Conversation

@TaprootFreak

@TaprootFreak TaprootFreak commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Problem

Between 2026-06-08 13:12 UTC and 2026-06-15 10:30 UTC, approximately 3 969 `FinancialDataLog` rows were persisted with `valid = true` while carrying a `totalBalanceChf` well above the expected operating-equity band (20–30 k CHF).

The spike was caused by a transient accounting event: `plusBalanceChf` rose by > 200 k CHF while large orders were pending. Once the baseline had shifted to that elevated level, subsequent ticks produced balance deltas that fell within `financeLogTotalBalanceChangeLimit` (5 000 CHF), so the invalid flag was never triggered — even though the absolute total was implausible.

Verified on 2026-06-15 via `/gs/debug`:
```sql
SELECT COUNT(*), MIN(created), MAX(created)
FROM log
WHERE subsystem = 'FinancialDataLog'
AND created >= NOW() - INTERVAL '7 days'
AND (message::jsonb -> 'balancesTotal' ->> 'totalBalanceChf')::numeric > 50000
AND valid = true;
-- → 3 969 rows | 2026-06-08T13:12:39Z – 2026-06-15T10:30:34Z
```

Solution

One-shot data migration that retroactively sets `valid = false` on affected rows.

Threshold: `totalBalanceChf > 50 000 CHF` — conservative cut-off well above the historical normal band (~20–30 k) and well below the spike peak (~430 k CHF).

Design decisions

Decision Rationale
SELECT ids first, then UPDATE by id Follows repo pattern (RepointAccountWalletForMergeMail); acts as an env-guard — no-op when rows don't exist (staging/dev)
Hard date range with exact earliest timestamp Prevents future accidental re-runs from touching newer data
down() best-effort Per-row original flag not stored; documented limitation. Selects same window/threshold but may over-restore rows already invalid before up() ran

Checklist

  • Migration only — no entity/schema changes, no config changes, no DTO changes
  • Env-guarded: no-op on staging/dev (no rows match the date range)
  • Exact ID-based UPDATE in up() — no table scan drift
  • down() implemented and limitation documented
  • Affected row count verified before writing migration
  • prettier --check passes
  • No console.log / debug statements

Between 2026-06-08 and 2026-06-15 ~3 969 FinancialDataLog rows were
stamped valid=true while carrying a totalBalanceChf > 50 000 CHF — well
above the expected operating-equity band of 20–30 k CHF.

Add a one-shot data migration that retroactively sets valid=false on
those rows so monitoring dashboards and anomaly alerts reflect the
correct state of the period.
@TaprootFreak TaprootFreak marked this pull request as ready for review June 15, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants