Skip to content

[#322] Fix backfill: missing title, silent data loss, cron auth#325

Merged
realproject7 merged 2 commits intomainfrom
task/322-fix-backfill
Mar 18, 2026
Merged

[#322] Fix backfill: missing title, silent data loss, cron auth#325
realproject7 merged 2 commits intomainfrom
task/322-fix-backfill

Conversation

@realproject7
Copy link
Copy Markdown
Owner

Summary

  • Missing title (E3): Added title: title || "" to processPlotChained() DB row, matching the direct indexer
  • Silent data loss (E6/E7): IPFS fetch failures and hash mismatches now logged to new backfill_failures table instead of silently skipping; failure count included in response JSON
  • Cron auth (D2): verifyCron() now rejects requests when CRON_SECRET is unset in production (NODE_ENV === "production")
  • Migration: supabase/migrations/00015_backfill_failures.sql creates the table with an index on storyline_id
  • Types: Added backfill_failures to Database interface in lib/supabase.ts

Fixes #322

Test plan

  • npm run build passes
  • npm run typecheck passes
  • Verify backfill cron inserts title for new plots
  • Verify IPFS failures are logged to backfill_failures table
  • Verify cron rejects unauthenticated requests in production
  • Run migration on Supabase

- Add title field to processPlotChained() DB row (matching direct indexer)
- Log IPFS fetch/hash-mismatch failures to backfill_failures table
  instead of silently skipping events
- Include failure count in response JSON
- Fail closed on missing CRON_SECRET in production
- Add Supabase migration for backfill_failures table
- Add backfill_failures type to Database interface

Fixes #322

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@project7-interns project7-interns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

T2b APPROVE

Verified all 3 acceptance items:

  1. Title field (E3): title: title || "" added to processPlotChained() PlotInsert, destructured from decoded event args — matches deployed contract ABI
  2. Failure logging (E6/E7): Both processStorylineCreated and processPlotChained now log IPFS/hash failures to backfill_failures table via logBackfillFailure(). Failure count included in response JSON. Helper function catches its own DB errors (no throw) — won't break the backfill loop
  3. Cron auth (D2): verifyCron() returns process.env.NODE_ENV !== "production" when no secret — correct fail-closed in prod, permissive in dev

Migration is clean with storyline_id index. TypeScript types match schema. Single commit, well-scoped.

Copy link
Copy Markdown
Collaborator

@project7-interns project7-interns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: REQUEST CHANGES

Summary

The main backfill fixes are in the right area, but the new failure-logging path still allows permanent silent loss when the backfill_failures insert itself fails. That leaves the audit issue unresolved in an important edge case.

Findings

  • [high] Failure logging errors are swallowed, but the cursor still advances.
    • File: src/app/api/cron/backfill/route.ts:54
    • Suggestion: Make logBackfillFailure() propagate insertion errors, or otherwise stop cursor advancement when a failure row cannot be recorded. As written, if the migration is missing or Supabase insert fails transiently, the event is neither indexed nor recorded, and last_block still advances at src/app/api/cron/backfill/route.ts:195, recreating the permanent data-loss path from audit #320.

Decision

Request changes. The PR should only advance the cursor after failed events are durably recorded, otherwise the new table does not actually guarantee recoverability.

If the backfill_failures insert itself fails, throw so the outer
catch handles it and increments the error count — prevents silent
data loss when the table is missing or DB is unreachable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@project7-interns project7-interns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE

Summary

The follow-up fixes the remaining failure-path bug: backfill now fails visibly if backfill_failures cannot be written, instead of silently advancing the cursor. The PR now satisfies the #322 acceptance criteria.

Findings

  • [low] The prior blocker is resolved because logBackfillFailure() now throws on insert errors, so failed events are either durably recorded or counted in the outer errors path.
    • File: src/app/api/cron/backfill/route.ts:54
    • Suggestion: None.

Decision

Approve. Backfilled plots now persist title, failed IPFS/hash events are recorded via the new table, production cron auth fails closed without CRON_SECRET, and local npm run build plus npm run typecheck both pass.

@realproject7 realproject7 merged commit c95a9ad into main Mar 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[#320] Fix backfill: missing title, silent data loss, cron auth

2 participants