Skip to content

Add portable SQLite storage and migrations - #43

Merged
kcsfelty merged 4 commits into
mainfrom
agent/portable-sqlite-migrations
Jul 21, 2026
Merged

Add portable SQLite storage and migrations#43
kcsfelty merged 4 commits into
mainfrom
agent/portable-sqlite-migrations

Conversation

@kcsfelty

@kcsfelty kcsfelty commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

What changed

  • preserves the existing D1 declarations and runtime classes while adding the neutral Sqlite*Like capability names, optional SqliteFirstCapability, and neutral quad aliases
  • adds the isolated @gnolith/diamond/node-sqlite subpath with a Node 22.16+ runtime guard, same-connection statement ownership, FIFO connection mutex, BEGIN IMMEDIATE ordered batches, rollback, WAL/foreign-key/busy-timeout setup, and explicit close/dispose lifecycle
  • adds the package-neutral _gnolith_migrations ledger with namespaced IDs, SHA-256 checksums, atomic ledger/application batches, race convergence, and explicit adoption records
  • makes initializeStore() migrate or conservatively adopt Diamond's exact existing RDF schema without replaying destructive DDL
  • bumps the planned feature release to 0.4.0 and documents embedded/file/memory/concurrency/migration behavior

Why

Headless and Docker-hosted Gnolith processes need Diamond's existing atomic SQLite semantics without Cloudflare provisioning, a database server, or Node imports leaking into Worker consumers. Higher-level packages still own their schemas, while Diamond owns the shared low-level adapter and migration ledger capability.

Validation

  • npm run check
  • 139 tests pass; 95.83% statement and 90.99% branch coverage
  • real Miniflare/workerd D1 migration matrix: fresh, repeat, legacy adoption/data preservation, partial rejection, drift/unknown history, rollback/recovery, racing initialization, namespace isolation
  • Node adapter: memory and file/reopen, rollback, concurrent writes, second-process busy timeout, cross-connection/use-after-close rejection
  • exact packed package consumer imports root, endpoint, and node-sqlite, persists/reopens a file, bundles/runs the Worker path, and asserts no node:sqlite leakage

Closes #40
Closes #41
Closes #42

@kcsfelty

Copy link
Copy Markdown
Collaborator Author

Addressed the independent review findings in 88fd5db:

  • legacy adoption now inspects the complete SQLite catalog and rejects any unexpected index, trigger, or view targeting or referencing Diamond tables, including unrelated-name triggers
  • Node runtime guard now models release lines exactly: 22.16+, 23.11+, or 24+
  • every ArrayBuffer/view binding is copied at bind time
  • the ledger must match the exact STRICT CREATE schema and constraints, not only table_info
  • emitted Node declarations no longer expose the raw connection or executor
  • INSERT/UPDATE/DELETE RETURNING retain positional rows and report meta.changes from a total_changes() delta

Validation: full npm run check passes with 156 tests, 96.19% statements, and 91.75% branches. Real workerd D1 includes adversarial trigger and non-STRICT ledger probes; exact-package smoke asserts the Node declaration has no connection/executor escape hatch.

@kcsfelty

Copy link
Copy Markdown
Collaborator Author

Re-review index bypasses are fixed in 39beef2. Every expected index now requires a catalog row whose tbl_name is exactly rdf_quads; explicit indexes must also match the canonical CREATE INDEX DDL, so partial predicates, sort/collation changes, and other semantic drift fail closed. Regression fixtures prove wrong-table same-name/same-columns and correct-table partial indexes are rejected without adoption. Full npm run check passes: 159 tests, 96.19% statements, 91.73% branches.

@kcsfelty
kcsfelty marked this pull request as ready for review July 21, 2026 12:23
@kcsfelty
kcsfelty merged commit 326da7e into main Jul 21, 2026
10 checks passed
@kcsfelty
kcsfelty deleted the agent/portable-sqlite-migrations branch July 21, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant