Conversation
go get github.com/jackc/pgx/v5/stdlib Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Create schema_pg.sql with: - search_fts TSVECTOR column on messages (ALTER TABLE IF NOT EXISTS) - GIN index for fast full-text search Unlike SQLite's separate FTS5 virtual table, PostgreSQL stores the tsvector directly on the messages table, eliminating the need for a JOIN during search queries. Update embed directive to include schema_pg.sql. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full PostgreSQL dialect implementation:
- Rebind: converts ? to $1,$2,... (handles quoted strings correctly)
- Now: NOW() instead of datetime('now')
- InsertOrIgnore: strips OR IGNORE, relies on InsertOrIgnoreSuffix
to append ON CONFLICT DO NOTHING
- UpdateOrIgnore: strips OR IGNORE (PG raises on conflict)
- FTS: tsvector column on messages with weighted search
(subject='A', from='B', body/to/cc unweighted), GIN index,
plainto_tsquery for search, ts_rank for ordering
- FTSAvailable: checks information_schema for search_fts column
- FTSNeedsBackfill: MAX(id) with NULL check on search_fts
- InitConn: sets statement_timeout
- SchemaStaleCheck: information_schema instead of pragma_table_info
- Error codes: 42701 (duplicate_column), 23505 (unique_violation),
42P01 (undefined_table) via pgconn.PgError
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Open() and OpenReadOnly() now detect postgres:// and postgresql:// URLs and use PostgreSQLDialect + pgx driver. SQLite path remains the default. PostgreSQL connection pool settings: - SetMaxOpenConns(25) vs SQLite's 4 - SetMaxIdleConns(5) Read-only mode sets default_transaction_read_only = ON for safety. Register pgx driver via side-effect import in dialect_pg.go. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Create a scaffolded PostgreSQLEngine parallel to SQLiteEngine. Implements the portable methods directly (GetMessage, GetMessageBySourceID, GetAttachment, ListAccounts, GetTotalStats for non-search cases) using PostgreSQL-specific SQL with $N placeholders. Methods that depend on SQLite-specific constructs (strftime, FTS5 MATCH, sqlite_master) return ErrNotImplemented pending follow-up work to parameterize the shared query-building helpers in sqlite.go. This provides a foundation for PostgreSQL read-path support that can be extended incrementally. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
testutil.NewTestStore() now detects MSGVAULT_TEST_DB env var: - unset/empty: SQLite (default, unchanged) - postgres://... URL: PostgreSQL with per-test schema isolation PostgreSQL test setup creates a random schema (msgvault_test_<hex>) via setup connection, then opens the store with search_path pointing to that schema. Cleanup drops the schema CASCADE on test end. Also add dialect_pg_test.go with table-driven tests for: - PostgreSQLDialect.Rebind (including quoted-string handling) - PostgreSQLDialect.Now, InsertOrIgnore, InsertOrIgnoreSuffix - PostgreSQLDialect.FTSSearchClause (paramIndex handling) - SQLiteDialect.FTSSearchClause (equivalence) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Correctness fix: - PostgreSQLDialect.InsertOrIgnore now auto-appends ON CONFLICT DO NOTHING for complete statements (prefix-only callers still use InsertOrIgnoreSuffix). This ensures idempotency at call sites like UpsertReaction, EnsureConversationParticipant, EnsureParticipantsBatch where the full INSERT statement is passed in one piece. Documentation: - Add pg_refactor_docs/PG_STATUS.md spelling out what works, what doesn't, and the blockers that must resolve before PostgreSQL is functional end-to-end (schema type translation, Rebind threading, LastInsertId replacement, query engine parameterization). - Add make test-pg target for running tests against PostgreSQL. - Document MSGVAULT_TEST_DB and schema_pg.sql in CLAUDE.md. PR2 is explicit scaffolding: the Dialect interface and PostgreSQLDialect are validated, but substantial follow-up work is needed to make the store layer and query engine actually usable with PostgreSQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wrap raw *sql.DB with newLoggedDB() in openPostgres and openPostgresReadOnly, matching the SQLite paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Upstream PR1 (merged as wesm#276) expanded the Dialect interface during review: FTSUpsert replaces FTSUpsertSQL (dialect owns argument shape), FTSSearchClause now returns `?` placeholders with an orderArgCount so loggedDB can rebind uniformly, FTSAvailable/FTSNeedsBackfill no longer return error, and four new methods (InsertOrIgnorePrefix, FTSRebuildSchema, IsBusyError, plus the newLoggedDB rebind argument) join the contract. Update PostgreSQLDialect to implement the new shape. FTSRebuildSchema returns an "unimplemented" error for now — proper tsvector rebuild is deferred to PR3 alongside the rest of the functional work. Remove the dead UpdateOrIgnore helper (not in the interface, no callers). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
roborev: Combined Review (
|
Three fixes from the combined review; the high finding (Makefile test-pg target) was already resolved in 1596d41 — verified with `make -n test-pg`. - internal/store/store.go: openPostgresReadOnly now sets default_transaction_read_only via pgx RuntimeParams + stdlib RegisterConnConfig, so the parameter is sent in the startup packet of every pooled connection. The previous `db.Exec("SET ...")` only affected whichever single connection happened to serve the Exec — later operations on a different pooled connection ran writable. - internal/query/postgres.go (GetMessage / GetMessageBySourceID): the partial implementation only fetched scalar fields from messages and message_bodies. The Engine contract (matched by SQLite/DuckDB) also populates participants, labels, attachments, ReceivedAt, and a raw-MIME body fallback. Returning a partially-populated MessageDetail silently violated the contract for callers (cmd/show_message, api/handlers, mcp/handlers, etc.). Per the review's preferred fix, return ErrNotImplemented until the full message-detail path lands; drop the now-unused getMessageByQuery + rebindPg helpers. - internal/query/postgres.go (GetTotalStats): attachment counts and sizes ignored HideDeletedFromSource, WithAttachmentsOnly, the message_type='email' restriction, and only filtered by source_id. Build attachment stats by joining messages and reusing the same WHERE clause as the message-stats query so deleted/non-email messages no longer leak into the totals. docs/PG_STATUS.md: mark blocker wesm#8 (read-only enforcement) resolved and update the "What works" entry to describe the RuntimeParams path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
roborev: Combined Review (
|
`argIdx` was incremented after the only usage point, leaving the final write dead. golangci-lint (ineffassign) flagged this. Switch to the standard `len(args)+1` pattern, which is lint-clean and stays extensible if future conditions append more parameters. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PG dialect work added github.com/jackc/pgx/v5 as a direct dependency, which changes the vendored module set hash. CI's nix-build job reported the new expected hash; apply it here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
roborev: Combined Review (
|
|
Thank you, I will review this and merge soon |
PR2 of the 4-PR PostgreSQL backend split (PR1 merged as #276).
Summary
PostgreSQLDialectimplementing theDialectinterface.pgxdriver intostore.Open()forpostgres://URLs.schema_pg.sqlwith a tsvector FTS column and GIN index.PostgreSQLEnginescaffold parallel toSQLiteEngine.MSGVAULT_TEST_DB(make test-pg).PostgreSQLDialectwith the post-review PR1 interface (FTSUpsert, ?-placeholder FTSSearchClause, InsertOrIgnorePrefix, FTSRebuildSchema, IsBusyError).Status
PostgreSQL is not functionally usable after this PR — see
docs/PG_STATUS.mdfor the full punch list. The blockers (schema type translation,Rebindthreading,LastInsertId→RETURNING, query-engine parameterization) land in PR3.SQLite remains the default and only production-ready backend; SQLite test suite is unchanged.