feat: move integration freshness sync onto Inngest - #20
Merged
Conversation
The scheduled poll ran as a Vercel cron that grabbed a singleton lease row and then chained POSTs to itself, hop by hop, until every due connection had had its turn. The lease existed because two overlapping chains would double the provider quota and race the same watermark; the hops existed because one serverless invocation could not outlive the work. Inngest already solves both. A cron function lists the due connections and sends one event each; a per-connection function does the actual poll. Retries, the concurrency cap that keeps a provider from being hammered, and the failure hook are declarations on the function rather than logic in the route. Observable behaviour is unchanged: same 04:00 schedule, same selection order, same backoff tiers, same watermark semantics. Deleted with the mechanism they served: `IntegrationSyncLease`, the `/api/cron/sync-integrations` route and its guard, the `crons` entry in `vercel.json`, and `CRON_SECRET` — nothing reads it once the last cron route is gone. Functions are feature-owned and collected in `src/server/inngest.ts`, the composition root for background work, mirroring how `api/root.ts` collects tRPC routers. `src/lib/` may not import from `src/features/`, so the collection cannot live beside the client. Closes #10 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
5 tasks
`refreshTokenEncrypted` and `tokenExpiresAt` were write-only: set once in the OAuth callback and read by nothing. Both were always NULL besides — Notion's `exchangeCode` returns neither, and Notion is the only provider in the registry. `BaseIntegrationProvider.refreshToken` was a stub whose entire behaviour was to throw, with no caller. Notion access tokens do not expire; they die when a user revokes access, and no refresh call fixes that. A failing poll already retries and then backs off, which is the right outcome for a connection that needs reconnecting by hand. A provider that genuinely needs a refresh path can add these back together with the code that reads them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10.
The scheduled freshness poll ran as a Vercel cron that took a singleton lease row and then chained POSTs to itself, hop by hop, until every due connection had had its turn. The lease was there because two overlapping chains would double the provider quota and race the same watermark; the hops were there because one serverless invocation could not outlive the work.
Inngest already solves both, so the mechanism goes and the domain rules stay.
Shape
integration-sync— cron0 4 * * *. Lists due connections, sends onescibly/integration-poll.requestedper connection. Nothing else.integration-poll— one run per connection. Refreshes, polls the provider for pages modified since the watermark, marks matching sources stale, advances the watermark. Throws on failure, because the throw is what Inngest retries.retries: 2, so three attempts.concurrency: { key: "event.data.provider", limit: 3 }— the quota ceiling, previously implicit in the chain being serial.onFailurerecords the failure and applies the backoff once, after the retries are spent, not once per attempt.The event carries
{ connectionId, provider }and no credential.Deleted
IntegrationSyncLease(+ migration),/api/cron/sync-integrationsand its route guard, thecronsentry invercel.json, andCRON_SECRET— nothing reads it once the last cron route is gone. Net −800 lines.Behaviour
Unchanged, deliberately: same 04:00 schedule, same least-recently-attempted-first selection, same lapsed-subscription exclusion, same backoff tiers, same watermark semantics (the watermark takes the instant the poll started, so an edit made mid-poll is caught by the next one rather than missed).
Verified against a real Notion workspace
Beyond the unit tests, run end to end on a live dev stack:
consecutiveFailuresclearedlastPolledAt − 1min, returned nothing, leftstaleAtaloneonFailurerun →consecutiveFailures+1 (not +3),nextPollAfterat the exact tier,lastPolledAtuntouchedlastAttemptedAtonly, never contacted the providerNotes for review
apps/app/src/server/inngest.ts, the composition root for background work, the wayapi/root.tscollects tRPC routers. Thearchitecture/boundariesrule forbidssrc/lib/importingsrc/features/, so the collection cannot sit beside the Inngest client.CRON_SECRET. It was only ever read by the route this PR deletes. Say the word and I'll put it back.refreshTokenEncryptedandtokenExpiresAtwere write-only — set in the OAuth callback, read nowhere — and always NULL, since Notion'sexchangeCodereturns neither and Notion is the only provider in the registry.BaseIntegrationProvider.refreshTokenwas a stub whose whole behaviour was to throw, with no caller. Notion tokens don't expire; they die on revocation, which no refresh fixes, and the retry-then-back-off path already handles that correctly. Whoever adds a provider that needs a refresh adds these back with the code that reads them.🤖 Generated with Claude Code