Skip to content

feat: SQL parser vs Proxy#16

Merged
digitalmio merged 7 commits into
mainfrom
feat-/sql-parser-attempt-2
May 16, 2026
Merged

feat: SQL parser vs Proxy#16
digitalmio merged 7 commits into
mainfrom
feat-/sql-parser-attempt-2

Conversation

@digitalmio
Copy link
Copy Markdown
Owner

No description provided.

Create `createTestDb` and `createMockDOStorage` helpers to support
testing with better-sqlite3 and Drizzle ORM.
Extract `recordMutationWithCascades` into a new `createTrackedClient.ts`
module as `recordCascades`, and add `createTrackedClient` function to
wrap DurableObjectStorage with SQL tracking via Proxy. Include
comprehensive test coverage for cascade propagation and table tracking.
Table tracking has been moved to the client-level SQL proxy
(`createTrackedClient`), which is more reliable and captures all
operations regardless of how the ORM is called. This eliminates
redundant tracking logic and simplifies the proxy signatures.

Also add TypeScript type casts to fix type inference issues in
`mockDOStorage.ts` and remove unused test fixtures.
Expose raw database instance in test setup and add comprehensive test
suite verifying that unsafeRawDb operations are tracked for read/write
access while bypassing safety enforcement constraints.
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 16, 2026

Greptile Summary

This PR replaces proxy-layer table tracking (via getTableName + recordMutationWithCascades) with SQL-parser-based tracking that intercepts DurableObjectStorage.sql.exec() calls and uses sqlite3-parser to extract read/write tables and WHERE param bindings.

  • parseSqlTracking.ts parses every SQL statement to classify query type, collect table references (including CTEs, JOINs, subqueries, UNIONs), and extract WHERE-clause parameter bindings (whereIds); WHERE scoping is now correctly limited to WHERE clauses rather than the entire AST.
  • createTrackedClient.ts wraps DurableObjectStorage with a Proxy that calls parseSqlTracking on every sql.exec invocation and populates tablesRead/tablesWritten; the cascade graph is now correctly threaded here instead of being passed an empty Map.
  • createTrackedDb.ts now wires createTrackedClient into realDb.session.client at startup, simplifying mutation proxies (which no longer carry table-tracking responsibilities) and emitting console.warn when the wiring path is unavailable.

Confidence Score: 5/5

Safe to merge — the architectural shift from proxy-layer to SQL-parser-based tracking is well-tested by both unit and integration tests, cascade handling is now correct, and the old silent tracking failures now emit warnings.

All changed paths are covered by tests that exercise the new SQL-interception path end-to-end. The simplification of mutation proxies (dropping table-name coupling) is clean. The one non-blocking finding — whereIds being computed but not yet consumed — does not affect correctness of any current feature.

No files require special attention beyond the whereIds dead-computation note in createTrackedClient.ts.

Important Files Changed

Filename Overview
packages/server/src/tools/parseSqlTracking.ts New SQL parser using sqlite3-parser; extracts query type, table reads/writes, and WHERE param bindings. WHERE scoping now correctly limited to WHERE clauses (not JOIN ON or HAVING). Well-tested; 217 lines, within the updated 250-line hard limit.
packages/server/src/tools/createTrackedClient.ts New proxy wrapping DurableObjectStorage.sql.exec to track table reads/writes via parseSqlTracking; whereIds is computed but not yet consumed here.
packages/server/src/tools/createTrackedDb.ts Replaced proxy-layer table tracking (getTableName + recordMutationWithCascades) with client-level SQL interception via createTrackedClient; now emits console.warn when wiring fails instead of silently skipping. Cascade graph is now correctly threaded to the SQL layer.
packages/server/src/tools/createMutationProxy.ts Simplified — table tracking params (tableName, tablesWritten) removed; proxy now only enforces WHERE-required and .prepare() blocks since tracking moved to SQL layer.
packages/server/src/tools/recordMutation.ts Deleted; recordMutationWithCascades renamed to recordCascades and relocated to createTrackedClient.ts, where it now receives the real cascadeGraph instead of an empty Map.
packages/server/src/test-utils/mockDOStorage.ts New test utility implementing a DurableObjectStorage mock backed by better-sqlite3; supports sql.exec, transactionSync, and databaseSize.
packages/server/src/tools/proxy.integration.test.ts New integration test suite covering limit enforcement, WHERE enforcement, table tracking via the client proxy, insert chaining, prepare blocking, and unsafeRawDb passthrough — good coverage of the new tracking architecture.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant createTrackedDb (Proxy)
    participant Drizzle session
    participant createTrackedClient (Proxy)
    participant parseSqlTracking
    participant DurableObjectStorage.sql.exec

    Caller->>createTrackedDb (Proxy): db.insert/update/delete/select(...)
    createTrackedDb (Proxy)->>Drizzle session: enforce WHERE / limit, delegate to real builder
    Drizzle session->>createTrackedClient (Proxy): session.client.sql.exec(sql, ...params)
    createTrackedClient (Proxy)->>parseSqlTracking: parseSqlTracking(sql, params)
    parseSqlTracking-->>createTrackedClient (Proxy): { tablesRead, tablesWritten, whereIds }
    createTrackedClient (Proxy)->>createTrackedClient (Proxy): tablesRead.add / recordCascades(tablesWritten)
    createTrackedClient (Proxy)->>DurableObjectStorage.sql.exec: forward original call
    DurableObjectStorage.sql.exec-->>Caller: result
Loading

Reviews (2): Last reviewed commit: "Refactor SQL tracking to only extract WH..." | Re-trigger Greptile

Comment thread packages/server/src/tools/parseSqlTracking.ts
Comment thread packages/server/src/tools/parseSqlTracking.ts Outdated
Comment thread packages/server/src/tools/createTrackedDb.ts
Restructured `parseSqlTracking` to explicitly collect WHERE clauses from
SELECT/UPDATE/DELETE statements before traversing them. This prevents
parameter extraction from JOIN ON and HAVING clauses, which should not
be treated as row-level scoping conditions.

Also relaxed the file size guideline in AGENTS.md (150–200 lines soft,
250 hard) and added console warnings when SQL tracking cannot be wired
due to missing `realDb.$client` or `realDb.session`.
@digitalmio digitalmio merged commit 1430db0 into main May 16, 2026
3 checks passed
@digitalmio digitalmio deleted the feat-/sql-parser-attempt-2 branch May 16, 2026 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant