Implement Idempotent Cursor Recovery on SIGTERM

[SHAURYASANYAL3]

🛑 The Problem

Cloud providers (Vercel, AWS, Kubernetes) routinely send SIGTERM signals to shut down worker instances. If an Inngest worker is halfway through syncing 20,000 commits and gets killed, it currently fails. Upon retry, it starts over from Page 1, creating an infinite loop of failures and wasted API calls.

💡 The Solution

We must make the ingestion loop idempotent and resumable by persisting the pagination cursor (e.g., the timestamp of the last processed commit) to the database.

🛠️ Implementation Details

1. Add a last_synced_cursor string column to the repositories table.
2. During the async generator fetch loop, update this column in Postgres every 1,000 commits.
3. When the Inngest worker boots up, check the DB for an existing cursor.
4. If found, pass the cursor to the GitHub API (e.g., using GraphQL after or REST since parameters) to resume exactly where it died.

✅ Acceptance Criteria

• The worker can safely survive a hard crash and resume without duplicating data.
• Network efficiency is maximized (no re-fetching old pages).

---

Ready to tackle this? Comment .take below to get automatically assigned!