feat: PostgreSQL driver integration + ANALYZE API + multi-hop perf#95
Merged
Conversation
`createPostgresBackend` now officially supports four Drizzle PostgreSQL adapters: `node-postgres`, `postgres-js`, `neon-serverless` (WebSocket), and `neon-http` (auto-detected; transactions disabled since HTTP can't hold a session). The pg + postgres-js paths are exercised end-to-end by the existing adapter and integration test suites (~250 tests each); the two Neon paths get wiring smoke tests covering driver detection, fast-path routing, type coercion, and capability surface. Two perf wins on PostgreSQL came from this work: - Server-side prepared statements on the node-postgres / neon-serverless fast path. Each unique compiled SQL string gets a stable counter-based statement name so PostgreSQL caches the plan after first execution. Combined with bypassing Drizzle's session wrapper for raw SQL execution, this drops the 3-hop benchmark from ~7.5ms to ~0.8ms median (9x), bringing TypeGraph-on-PostgreSQL to parity with Neo4j 5.26 on every single-query and multi-hop shape we measure. - `store.refreshStatistics()` / `backend.refreshStatistics()` API that runs ANALYZE on the TypeGraph-managed tables. Without it the planner works from stale post-bulk-load stats — 5ms forward traversal instead of 0.5ms — until autovacuum catches up. Both SQLite and PostgreSQL backends implement it. Other notable changes: - New `PostgresBackendOptions.capabilities?: Partial<BackendCapabilities>` for explicit capability overrides (e.g., custom HTTP-style drivers). - `pg`, `postgres`, and `@neondatabase/serverless` declared as optional peer deps so installs surface the driver choice clearly. - Edge-runtime safety: the postgres execution adapter has no static `node:*` imports, so the `@nicia-ai/typegraph/postgres` entry point loads cleanly under Cloudflare Workers / Vercel Edge. - Identifier escaping in `refreshStatistics` uses `quoteIdentifier` + `sql.join` so user-supplied table names round-trip safely. - Benchmark harness gains `--postgres-driver=pg|postgres-js` and runs ANALYZE post-seed for steady-state measurement. - Documentation in `apps/docs/src/content/docs/backend-setup.md` covers all four drivers with install + setup snippets and a runtime-to-driver matrix. Type surface change: `GraphBackend` now requires `refreshStatistics(): Promise<void>`. `TransactionBackend` excludes it (stats refresh isn't meaningful inside a transaction). External implementations (uncommon) need a no-op or proper implementation.
pdlug
force-pushed
the
feat/postgres-driver-integration
branch
from
85a82ad to
a948975
Compare
Doc and behavior fixes for the no-transactions fallthrough, the new refreshStatistics() API, and the prepared-statement cache. Behavior: - LRU-bound the module-scope statement-name cache in the PostgreSQL fast path (default 256, parity with the SQLite adapter). High- cardinality SQL text — variable-length IN-list expansions, custom backend.execute() calls, generated aliases — would otherwise grow both the JS Map and per-session prepared-statement memory inside PostgreSQL without bound. The monotonic counter is independent of cache size so eviction never recycles a name; recycling could collide with a still-prepared statement on a long-lived pg connection. - Add prepareStatements: false opt-out on createPostgresBackend for pgbouncer transaction-pool mode (named statements registered on one backend connection aren't visible on the next). - Extract a shared getOrCreateLru<K, V> helper used by both the PostgreSQL statement-name cache and the SQLite prepared-statement cache; the duplicated eviction logic now lives in one place. Docs: - Document the no-transactions fallthrough on store.transaction() and store.batch() JSDocs — the behavior is deliberate and tested, but the public docstrings still claimed unconditional atomicity / snapshot consistency. - Replace the limitations.md and schemas-stores.md sections that said store.transaction() throws ConfigurationError on D1; the toplevel store now falls through to sequential execution on any backend that reports transactions: false. - Replace the raw "ANALYZE typegraph_nodes, ..." snippet in backend-setup.md with the new store.refreshStatistics() API; the public API handles custom table names and the SQLite path correctly. - Fix backend.capabilities.vectorSearch typo (actual API is backend.capabilities.vector?.supported). - Fix store.bulkCreate(batch) snippet to the collection-scoped store.nodes.Document.bulkCreate(batch) in store.ts JSDoc and the changeset. - Add prepareStatements and preparedStatementCacheMax to the createPostgresBackend signature in the API reference.
pdlug
force-pushed
the
feat/postgres-driver-integration
branch
from
90b2918 to
5789afd
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
createPostgresBackend(db)entry point:node-postgres,postgres-js,@neondatabase/serverless(WebSocket Pool), and@neondatabase/serverless(HTTP). Driver detection, capability surface, and a fast-path execution adapter are all driver-aware.neon-httpis auto-detected andcapabilities.transactionsis set tofalseso transactional code paths fall through to sequential execution rather than throwing.store.refreshStatistics()/backend.refreshStatistics()API that runsANALYZEon the TypeGraph-managed tables. Without it the planner works from stale post-bulk-load stats — the difference between a 0.5ms and a 5ms forward traversal on PG, or a 0.9ms vs 23ms fulltext query on SQLite — until autovacuum catches up. Now a one-liner after any bulk import.node:*imports, so@nicia-ai/typegraph/postgresloads cleanly on Cloudflare Workers / Vercel Edge.GraphBackendnow requiresrefreshStatistics(): Promise<void>.TransactionBackendexcludes it. External implementations (uncommon) need a no-op or proper implementation.PostgresBackendOptions:capabilities?: Partial<BackendCapabilities>for explicit overrides on custom HTTP-style drivers.pg,postgres, and@neondatabase/serverlessdeclared so installs surface the driver choice clearly.