Migrate from supabase to generic postgres db#8
Merged
Conversation
- Add pg, @types/pg for PostgreSQL client - Add prisma, @prisma/client for schema management - Add @prisma/adapter-pg for Prisma 7 support
- Create prisma/schema.prisma with PostgreSQL provider - Define docs table schema matching existing structure - Use Unsupported type for pgvector vector(3072) column - Configure indexes and constraints
- Rename SupabaseConfig to DatabaseConfig - Update fields: databaseUrl, table, keep search config - Update AppConfig to use database instead of supabase
- Create database/types.ts with interfaces and types - Export ExistingChunkInfo, PostgresDocRow, RetrievedChunk - Define types for database operations
- Create database/prisma.ts for Prisma client initialization - Use @prisma/adapter-pg with pg.Pool for Prisma 7 - Create utils/getDatabaseUrl.ts for shared URL resolution - Support MIMIR_DATABASE_URL and DATABASE_URL
- Create database/client.ts with PostgresVectorStore class - Implement constructor, connection pool setup, verifyConnection() - Use pg.Pool for PostgreSQL connections - Replace Supabase client with direct PostgreSQL connection
- Create database/chunks.ts with chunk operations - Implement fetchExistingChunks, fetchChunksByChecksums - Implement upsertChunks, deleteChunksByIds, updateChunkOrders - Use parameterized SQL queries for safety
- Create database/moves.ts with moveChunksAtomic method - Handle two-phase atomic moves with temporary filepaths - Prevent unique constraint violations during moves
- Create database/orphans.ts with orphan detection methods - Implement findOrphanedChunkIds, findStrandedChunkIds - Handle batch fetching and repository scoping
- Create database/search.ts with search methods - Implement matchDocuments using match_docs SQL function - Implement searchDocumentsFullText using match_docs_bm25 - Format embeddings as PostgreSQL array strings
- Create database/index.ts to export all modules - Re-export types, client, and all operation modules - Provide clean public API for database operations
- Create initial migration with vector extension - Create docs table with all columns and indexes - Add match_docs and match_docs_bm25 SQL functions - Add updated_at trigger
- Create prisma.config.ts for Prisma 7 datasource URL - Use shared getDatabaseUrl utility - Configure schema and migrations paths
- Update loadConfig.ts to use MIMIR_DATABASE_URL - Remove Supabase-specific environment variables - Update all config.supabase references to config.database
- Update server.ts to use createPostgresStore - Update context.ts to use PostgresVectorStore type - Update all route handlers to use new store - Replace SupabaseVectorStore with PostgresVectorStore
- Update askAi.ts to use PostgresVectorStore - Update config references from supabase to database
- Update pipeline.ts to use PostgresVectorStore - Update ingest CLI and route handlers - Update all type references
- Update Makefile to use Prisma migrations instead of direct SQL - Update docker-entrypoint.sh to use Prisma migrate deploy - Remove Supabase-specific connection string construction - Support MIMIR_DATABASE_URL and DATABASE_URL
- Update README.md with PostgreSQL setup instructions - Update configuration.mdx to remove Supabase references - Add database configuration options (BM25, hybrid search) - Update getting-started and deployment docs - Clarify single vs multiple repo configuration
- Remove obvious comments that restate what code does - Keep only essential documentation and non-obvious logic comments - Clean up code for better readability
- Remove Supabase-specific variables - Use MIMIR_DATABASE_URL for generic PostgreSQL - Update all configuration examples - Add all single and multiple repo configuration options - Include all optional LLM and database settings
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.