Migrate infrastructure to AWS (SST + Fargate + RDS + Clerk + S3 + SES)#96
Closed
saratpediredla-level5 wants to merge 10 commits into
Closed
Migrate infrastructure to AWS (SST + Fargate + RDS + Clerk + S3 + SES)#96saratpediredla-level5 wants to merge 10 commits into
saratpediredla-level5 wants to merge 10 commits into
Conversation
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Translates the Supabase schema in backend/schema.sql to a Drizzle ORM definition targeting AWS RDS/Aurora Postgres. Lays the foundation for Stages D/E, which rewrite the routes and middleware off supabase-js. - backend/src/db/schema.ts: 16 tables translated 1:1 with schema.sql. user_id columns that previously referenced auth.users(id) are now plain text (Clerk owns identity). RLS policies, the handle_new_user trigger, and Supabase grants are not modelled - access control moves to the Express layer. - backend/drizzle.config.ts: drizzle-kit config with a dummy default DATABASE_URL so `generate` works without env setup. - backend/drizzle/0000_init.sql: initial migration with all tables, indexes (including GIN on shared_with), check constraints, and foreign keys. Prepended CREATE EXTENSION pgcrypto. Manually appended the two circular FKs (documents.current_version_id and document_edits.chat_message_id) that Drizzle could not emit inline. - backend/package.json: dropped @supabase/supabase-js; added drizzle-orm, drizzle-kit, pg, @types/pg, @clerk/backend. Added db:generate / db:migrate / db:push / db:studio scripts. The legacy backend/schema.sql is kept in place until upstream merges clear; Stage D will start removing supabase.ts and the route consumers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop @supabase/supabase-js from middleware/auth.ts; verify Clerk JWTs with @clerk/backend (verifyToken), supporting both JWKS and offline jwtKey verification. - First-request profile bootstrap (in-memory cache) replaces the old Supabase handle_new_user trigger. - Add backend/src/lib/db.ts exporting Drizzle client + pg.Pool + withTransaction helper. - Delete backend/src/lib/supabase.ts. Routes and several libs still import it; Stages E1/E2 will rewrite them. tsc --noEmit fails (42 errors), all confined to files Stage E1/E2 will rewrite. db.ts and middleware/auth.ts typecheck cleanly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrite user, downloads, workflows, and projectChat routes — plus the access, userSettings, and userApiKeys libs — to query Postgres directly via Drizzle instead of Supabase. Brings the backend tsc error count from 40 to 24; remaining errors are all in Stage E2 files (chat, projects, documents, tabular, chatTools, documentVersions). - Lib helper signatures keep a trailing optional `_db` arg so out-of-scope E2 callers don't all immediately break on extra-argument errors before Stage E2 lands. - Workflows route now backfills sharer display names via the Clerk Backend API instead of the Supabase admin listUsers call. - `/user/account` no longer calls supabase.auth.admin.deleteUser; it removes the local profile row and leaves Clerk-side deletion to a separate path. - userApiKeys keeps the AES-GCM crypto unchanged; only the DB plumbing swapped to Drizzle's onConflictDoUpdate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rewrites projects, documents, tabular, chat, chatTools, and
documentVersions to use Drizzle queries against the shared db
singleton. Drops the trailing _db?: unknown bridge parameter that
Stage E1 added to access/userSettings/userApiKeys helpers.
Behavioral notes:
- Member email->id lookups in projects.ts and tabular.ts now use
Clerk's users.getUserList({ emailAddress: [...] }) and
users.getUser(id) instead of Supabase's auth.admin.listUsers.
- chat.ts /chat OR-filter converted from Supabase's PostgREST .or()
string syntax to Drizzle's or(eq, inArray).
- Dead buildTabularContext helper removed from tabular.ts (no
callers anywhere).
- jsonb inserts retain as any casts to match Stage E1 pattern.
tsc --noEmit -p backend: 0 errors.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Rewrite backend/src/lib/storage.ts to use native AWS S3 via the default credential chain (Fargate task role). Drop R2 endpoint, forcePathStyle, and explicit access-key env vars. BUCKET reads S3_BUCKET_NAME with a transitional R2_BUCKET_NAME fallback. - Add backend/src/lib/email.ts wrapping @aws-sdk/client-sesv2 with a sendEmail helper that no-ops when SES_FROM_ADDRESS is unset. - Swap deps in backend/package.json: drop resend, add @aws-sdk/client-sesv2. Refresh package-lock.json. - No existing Resend call sites in backend/src — helper added for future use. tsc --noEmit -p backend: 0 errors. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces every frontend Supabase auth touchpoint with Clerk equivalents. ClerkProvider wraps the app, clerkMiddleware protects routes, and Clerk's hosted SignIn/SignUp components render at /login and /signup via catch-all segments. mikeApi reads the bearer token from window.Clerk.session, and hooks/components that previously called supabase.auth.getSession() now use useAuth().getToken() from @clerk/nextjs. AuthContext is removed; consumers read user/userId from Clerk's useAuth/useUser hooks directly. Notes: - @clerk/nextjs 7.3.3 requires react ~19.2.3; project pins 19.2.0, so npm install was run with --legacy-peer-deps. Runtime works correctly. - tagWIdsOnRenderedDom in DocxView refactored to accept a token argument since it runs outside React hooks. - tsc --noEmit -p frontend: clean. Lint: 104 problems (-1 vs baseline). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Author
|
Created against the wrong repo (intended for fork LevelFive-Studio/helix-tribune); closing and reopening on the fork. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Full migration from the upstream Cloudflare/Supabase stack to an AWS-native stack. Single branch, 9 logically distinct commits.
@opennextjs/cloudflare)@opennextjs/aws, driven bysst.aws.Nextjsbackend/Dockerfile)@supabase/supabase-js)@clerk/nextjs+@clerk/backend).user_idcolumns are nowtext(Clerk IDs) instead ofuuid.@aws-sdk/client-sesv2(helper added; no current call sites in upstream).envfilessst.config.tscovers VPC, RDS, S3, Fargate, Nextjs, and all secretsStage-by-stage commits
e3ee400— infra: SST scaffold + Fargate Dockerfilecea1546— db: Drizzle schema (16 tables) + initial migration; swapsupabase-jsdepfcd1bea— auth: Clerk JWT verification + Drizzle client; first-request profile bootstrap replaces thehandle_new_usertrigger55d189d— routes (E1): simpler routes +access/userSettings/userApiKeyslibs to Drizzle8d56028— routes (E2): heavy routes (projects/documents/tabular/chat) +chatTools(3284 lines) to Drizzle; backendtsc --noEmitreaches 0 errorsc599ba5— storage+email: R2 → S3 inbackend/src/lib/storage.ts; Resend → SES helper addede20c5e9— frontend auth:<ClerkProvider>,clerkMiddleware, catch-all/login+/signup, all 8supabase.auth.getSession()call sites + 12useAuthconsumers migrated; 6 dead Supabase files deleted01b10ca— frontend deploy:@opennextjs/cloudflare→@opennextjs/aws;wranglerremoved; Cloudflare-only scripts dropped2c0806a— docs: README + CLAUDE.md updated for the new stackNotable design decisions
@supabase/supabase-js-shaped adapter on top ofpgto minimize route diffs, but every callsite gets rewritten explicitly. Cleaner, type-safe, no leaky abstractions.Set<string>cache replaces the dropped Supabasehandle_new_usertrigger.users.getUserList({ emailAddress: [...] })replaces Supabase'sauth.admin.listUsersfor member lookups inprojects.tsandtabular.ts— more efficient than the old "list 1000 users and filter" pattern.backend/schema.sqlkept on disk as a no-op file to ease upstream merges (the real schema source is nowbackend/src/db/schema.ts).nixpacks.tomlandfrontend/open-next.config.tsleft in place (rewritten in H, but the file is still there) so upstream PRs to those paths still 3-way-merge.Verification
npx tsc --noEmit -p backend: 0 errorsnpx tsc --noEmit -p frontend: 0 errorsnpm run lint --prefix frontend: 104 problems — all pre-existing, none introduced by this PR@supabase/*imports anywhere inbackend/srcorfrontend/srcDeployment checklist (post-merge)
npm installat repo root (SST CLI)npx sst secret setfor every secret declared insst.config.ts: Clerk publishable/secret/JWT keys, Anthropic/Gemini/OpenAI keys,UserApiKeysEncryptionSecret,DownloadSigningSecret,SesFromAddressnpx sst deploy --stage productionbackend/drizzle/0000_init.sql(ornpm run db:migrate --prefix backendwithDATABASE_URLpointed at RDS)emailclaim if you want to skip the per-user Clerk API lookup on first requestapp.<domain>at the CloudFront distribution SST createsUpstream-fork strategy
infra/UPSTREAM.mddocuments how this fork should pull fromupstream/main. Expect conflicts on every upstream PR that touchesbackend/src/routes/*,backend/src/middleware/auth.ts,backend/src/lib/storage.ts, or the frontend auth files. Non-conflicting paths:infra/,backend/Dockerfile,sst.config.ts,backend/src/db/.Test plan
sst deployto a staging stage and walk through the golden path: sign up via Clerk, create a project, upload a DOCX, run a chat, run a workflow, run a tabular review, download a generated DOCX/usr/bin/soffice) by triggering a DOC→PDF conversionCLERK_JWT_KEYfor offline verification)🤖 Generated with Claude Code