Skip to content

test(ingestion): reorg simulation deduplication test#95

Merged
Miracle656 merged 1 commit into
Miracle656:mainfrom
Depo-dev:feat/reorg-dedup-test-79
May 30, 2026
Merged

test(ingestion): reorg simulation deduplication test#95
Miracle656 merged 1 commit into
Miracle656:mainfrom
Depo-dev:feat/reorg-dedup-test-79

Conversation

@Depo-dev
Copy link
Copy Markdown
Contributor

Summary

Adds a reorg simulation integration test that validates the ingestion layer is fully idempotent under replay conditions (Issue #79).

  • tests/integration/reorg.test.ts: 12 deterministic tests across 4 suites:
    1. Normal ingest: all records inserted on first pass
    2. Full replay: 0 new records on second identical pass, store size unchanged, no duplicates
    3. Skipped-ledger gap simulation: detects missing ledger, backfills without duplicating existing records, all data present after resume, consistent after aggressive overlapping replays
    4. upsertTransfers contract: always passes skipDuplicates:true, skips Prisma call for empty input, returns correct inserted count
  • @prisma/client is mocked at the PrismaClient constructor level so the real upsertTransfers code path is exercised with an in-memory store that mirrors the Postgres @unique(eventId) constraint
  • tsconfig.test.json: extends root tsconfig with rootDir:. so tests/ compiles alongside src/ with full jest types
  • package.json: adds tests/ to jest roots + testMatch; uses tsconfig.test.json via transform config

Also fixes two pre-existing build errors now surfaced by the stricter test compilation:

  • src/db.ts: add local toDisplayAmount helper (was used but never imported)
  • src/api.ts: fix type assertions on OData-projected transfer fields in CSV export

Acceptance criteria

  • Deterministic deduplication works (replay inserts 0 records)
  • Replay-safe ingestion confirmed (all records present, none missing after gap + resume)
  • No regression in existing ingestion pipeline (58 tests pass across 5 suites)

Test plan

  • Run: npx jest tests/integration/reorg.test.ts -- confirm 12/12 pass
  • Run: npx jest -- confirm no regression in existing passing test suites

Closes #79

Validates that upsertTransfers is fully idempotent under replay
conditions — no duplicate rows regardless of how many times the
same ledger page is replayed.

- tests/integration/reorg.test.ts: 12 deterministic tests covering
  (1) normal ingest inserts all records once, (2) full replay inserts
  0 records, (3) skipped-ledger gap detection + backfill without
  duplicates, (4) upsertTransfers always passes skipDuplicates:true;
  @prisma/client is mocked at the PrismaClient constructor level so
  the real upsertTransfers code path is exercised with an in-memory
  store that mirrors the Postgres @unique(eventId) constraint
- tsconfig.test.json: extends root tsconfig with rootDir:. so
  tests/ is compiled alongside src/ with full jest types
- package.json: adds tests/ to jest roots + testMatch; uses
  tsconfig.test.json via transform config (replaces deprecated globals)
- src/db.ts: add local toDisplayAmount helper (was used but never
  imported — pre-existing build error)
- src/api.ts: fix type assertions on OData-projected transfer fields
  in CSV export route (pre-existing strict TS errors now surfaced)
@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented May 30, 2026

@Depo-dev Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Copy link
Copy Markdown
Owner

@Miracle656 Miracle656 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tight, well-targeted dedup proof.

  • Mock strategy: replacing PrismaClient at constructor level lets the real upsertTransfers code path run against an in-memory Map<string, TransferRecord> that mirrors the Postgres @unique(eventId) constraint. Same code path the production indexer uses — exactly the right contract test.
  • _createManyCalls spy confirms upsertTransfers always invokes createMany with skipDuplicates: true (a structural guarantee, not just an observed behavior).
  • Skipped-ledger gap test is the sharpest one: ingest ledgers 1–3 and 5 (4 missing), then replay 1–5 → only the previously-missing ledger 4 events get added, no duplicates on the rest. That's the actual #79 acceptance criterion ("no dup rows after replay" + "no data loss after skip-then-resume").
  • The Prisma namespace stub (sql, join, empty) prevents the mock from breaking on raw-SQL helpers used elsewhere in db.ts.

The two side-fixes in src/db.ts and src/api.ts are real compile errors that surfaced once the stricter test tsconfig pulled them into scope:

  • toDisplayAmount was used in db.ts without an import — adding a local helper is the minimal fix.
  • The OData CSV-export type assertion was unsound after #92 landed; tightening it is the right call.

Both fixes are scoped to making the test pipeline green, transparent in the PR body. Acceptable scope creep.

Heads-up to next contributor: this PR adds tsconfig.test.json and the same roots/testMatch/transform jest config that #88 (still awaiting rebase) also wants to add. arandomogg will hit a conflict on those two files when they rebase — straightforward "keep both" resolution.

Merging. Closes #79.

@Miracle656 Miracle656 merged commit 59c56dc into Miracle656:main May 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reorg simulation tests

2 participants