test(ingestion): reorg simulation deduplication test#95
Conversation
Validates that upsertTransfers is fully idempotent under replay conditions — no duplicate rows regardless of how many times the same ledger page is replayed. - tests/integration/reorg.test.ts: 12 deterministic tests covering (1) normal ingest inserts all records once, (2) full replay inserts 0 records, (3) skipped-ledger gap detection + backfill without duplicates, (4) upsertTransfers always passes skipDuplicates:true; @prisma/client is mocked at the PrismaClient constructor level so the real upsertTransfers code path is exercised with an in-memory store that mirrors the Postgres @unique(eventId) constraint - tsconfig.test.json: extends root tsconfig with rootDir:. so tests/ is compiled alongside src/ with full jest types - package.json: adds tests/ to jest roots + testMatch; uses tsconfig.test.json via transform config (replaces deprecated globals) - src/db.ts: add local toDisplayAmount helper (was used but never imported — pre-existing build error) - src/api.ts: fix type assertions on OData-projected transfer fields in CSV export route (pre-existing strict TS errors now surfaced)
|
@Depo-dev Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Miracle656
left a comment
There was a problem hiding this comment.
Tight, well-targeted dedup proof.
- Mock strategy: replacing
PrismaClientat constructor level lets the realupsertTransferscode path run against an in-memoryMap<string, TransferRecord>that mirrors the Postgres@unique(eventId)constraint. Same code path the production indexer uses — exactly the right contract test. _createManyCallsspy confirmsupsertTransfersalways invokescreateManywithskipDuplicates: true(a structural guarantee, not just an observed behavior).- Skipped-ledger gap test is the sharpest one: ingest ledgers 1–3 and 5 (4 missing), then replay 1–5 → only the previously-missing ledger 4 events get added, no duplicates on the rest. That's the actual #79 acceptance criterion ("no dup rows after replay" + "no data loss after skip-then-resume").
- The
Prismanamespace stub (sql,join,empty) prevents the mock from breaking on raw-SQL helpers used elsewhere indb.ts.
The two side-fixes in src/db.ts and src/api.ts are real compile errors that surfaced once the stricter test tsconfig pulled them into scope:
toDisplayAmountwas used indb.tswithout an import — adding a local helper is the minimal fix.- The OData CSV-export type assertion was unsound after #92 landed; tightening it is the right call.
Both fixes are scoped to making the test pipeline green, transparent in the PR body. Acceptable scope creep.
Heads-up to next contributor: this PR adds tsconfig.test.json and the same roots/testMatch/transform jest config that #88 (still awaiting rebase) also wants to add. arandomogg will hit a conflict on those two files when they rebase — straightforward "keep both" resolution.
Merging. Closes #79.
Summary
Adds a reorg simulation integration test that validates the ingestion layer is fully idempotent under replay conditions (Issue #79).
Also fixes two pre-existing build errors now surfaced by the stricter test compilation:
Acceptance criteria
Test plan
Closes #79