feat(test): add tabular-storage contract invariant suite across all backends#510
Merged
Merged
Conversation
…riant Adds a new contract suite under packages/test/src/contract/tabular-storage/ with a single assertion: subscribeToChanges fires exactly once per write in commit order (three sequential puts → three events, v1→v2→v3). - types.ts: TabularStorageContractOpts / TabularStorageContractAssertion - runTabularStorageContract.ts: describe.skipIf wrapper + block dispatch - assertions/subscribeToChanges.ts: the invariant with polling timing logic - InMemoryTabularStorage.test.ts: wired up with supportsSubscriptions: true
Extends the contract suite with a round-trip assertion: a Float32Array written via put() must come back as a Float32Array (not a plain Array or Buffer) via get(). Wires InMemoryTabularStorage with supportsVectorColumns. - types.ts: adds VectorItemSchema, VectorItemPrimaryKeyNames, createVectorStorage factory, capabilities.supportsVectorColumns, and "vectorColumnFormat" to TabularStorageContractAssertion - assertions/vectorColumnFormat.ts: new assertion block - runTabularStorageContract.ts: dispatches vectorColumnFormatBlock - InMemoryTabularStorage.test.ts: wired with supportsVectorColumns: true
…contract Asserts that when a withTransaction callback throws, none of its writes are visible afterward and any pre-transaction writes are untouched. - types.ts: adds capabilities.supportsTransactions and "withTransactionRollback" to TabularStorageContractAssertion - assertions/withTransactionRollback.ts: new assertion block - runTabularStorageContract.ts: dispatches withTransactionRollbackBlock - InMemoryTabularStorage.test.ts: wired with supportsTransactions: false (InMemory transactions are a no-op wrapper; skip rather than expect-fail)
Asserts that for any predicate, count() and query().length agree. Catches the common bug where count takes an optimizer fast path that disagrees with the row-returning query path (null handling, partial index usage, stale planner stats). - types.ts: adds "countMatchesQuery" to TabularStorageContractAssertion - assertions/countMatchesQuery.ts: new assertion block (no capability gate) - runTabularStorageContract.ts: dispatches countMatchesQueryBlock
…uite - Add runTabularStorageContract to: InMemory, Sqlite, Postgres, FsFolder, SharedInMemory, IndexedDb, Cached, Telemetry, and Supabase backends - Fix SqliteTabularStorage and PostgresTabularStorage to properly serialize TypedArrays as JSON arrays and reconstruct them on read; previously Float32Array round-tripped as plain integer-keyed objects - Add supportsQuery capability gate to skip countMatchesQuery on FsFolder (which throws StorageUnsupportedError on query()) - Add TYPED_ARRAY_CTORS lookup tables to Sqlite and Postgres providers for format-aware TypedArray reconstruction in sqlToJsValue https://claude.ai/code/session_01KJmJwgy4qsAK2iXXrC2yM4
@workglow/cli
@workglow/ai
@workglow/browser-control
@workglow/indexeddb
@workglow/javascript
@workglow/job-queue
@workglow/knowledge-base
@workglow/mcp
@workglow/storage
@workglow/task-graph
@workglow/tasks
@workglow/util
workglow
@workglow/anthropic
@workglow/bun-webview
@workglow/chrome-ai
@workglow/electron
@workglow/google-gemini
@workglow/huggingface-inference
@workglow/huggingface-transformers
@workglow/node-llama-cpp
@workglow/ollama
@workglow/openai
@workglow/playwright
@workglow/postgres
@workglow/sqlite
@workglow/supabase
@workglow/tf-mediapipe
commit: |
Polling-based subscriptions (FsFolder, IndexedDb) detect all writes that happen between two polls in a single snapshot diff, so the order in which they emit the resulting events is unspecified. Check that all three expected values are present (sorted) for polling implementations, and keep the strict write-order check only for event-driven backends. https://claude.ai/code/session_01KJmJwgy4qsAK2iXXrC2yM4
Coverage Report
File CoverageNo changed files found. |
Contributor
There was a problem hiding this comment.
Pull request overview
Introduces a reusable runTabularStorageContract invariant suite (subscribe, vector round-trip, transaction rollback, count==query) and wires it into all tabular storage backends. Also fixes a TypedArray serialization bug in SqliteTabularStorage and PostgresTabularStorage where Float32Array was being JSON-stringified as an integer-keyed object instead of an array, and was not reconstructed as a typed array on read.
Changes:
- New contract suite (
packages/test/src/contract/tabular-storage/) with capability-gated assertion blocks and sharedVectorItemSchema. - Backend test files (InMemory, Sqlite, Postgres, FsFolder, SharedInMemory, IndexedDb, Cached, Telemetry, Supabase) all invoke
runTabularStorageContractwith their capabilities. SqliteTabularStorage/PostgresTabularStoragenow serialize TypedArrays as JSON arrays and reconstruct them on read using the schemaformat: "TypedArray:*"annotation.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| providers/sqlite/src/storage/SqliteTabularStorage.ts | TypedArray JSON serialize/parse fix; reorders imports; minor formatting |
| providers/postgres/src/storage/PostgresTabularStorage.ts | TypedArray JSON serialize/parse fix using schema format; import reorder; minor formatting |
| packages/test/src/contract/tabular-storage/runTabularStorageContract.ts | New contract entry point composing four assertion blocks |
| packages/test/src/contract/tabular-storage/types.ts | Contract option/capability types and shared VectorItem schema |
| packages/test/src/contract/tabular-storage/assertions/subscribeToChanges.ts | Asserts subscription fires once per write, in commit order (event-driven) |
| packages/test/src/contract/tabular-storage/assertions/vectorColumnFormat.ts | Asserts Float32Array round-trip preserves type/values |
| packages/test/src/contract/tabular-storage/assertions/withTransactionRollback.ts | Asserts thrown callback rolls back writes |
| packages/test/src/contract/tabular-storage/assertions/countMatchesQuery.ts | Asserts count(p) == query(p).length across predicates |
| packages/test/src/test/storage-tabular/{InMemory,Sqlite,Postgres,FsFolder,SharedInMemory,IndexedDb,Cached,Telemetry,Supabase}TabularStorage.test.ts | Wire the new contract into each backend with appropriate capability flags |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runTabularStorageContract): four invariant blocks —subscribeToChanges,vectorColumnFormat,withTransactionRollback,countMatchesQuery— each gated by a capability flag (supportsSubscriptions,supportsVectorColumns,supportsTransactions,supportsQuery)SqliteTabularStorageandPostgresTabularStorage: both were serializingFloat32Arrayas integer-keyed JSON objects ({"0":0.1,...}) instead of arrays, then deserializing back as plain objects; now properly serializes to[0.1,...]arrays and reconstructs typed arrays on read using the schema'sformat: "TypedArray:*"annotationsupportsQuery: falsecapability gate added for FsFolder (itsquery()throwsStorageUnsupportedError)kb_idscope column in the PK can't conform to theCompoundSchemacontract type without a purpose-built scoped schemaTest plan
npx vitest run packages/test/src/test/storage-tabular/InMemoryTabularStorage.test.ts— all passnpx vitest run packages/test/src/test/storage-tabular/SqliteTabularStorage.integration.test.ts— all passnpx vitest run packages/test/src/test/storage-tabular/PostgresTabularStorage.integration.test.ts— all passnpx vitest run packages/test/src/test/storage-tabular/FsFolderTabularStorage.integration.test.ts— all passnpx vitest run packages/test/src/test/storage-tabular/SharedInMemoryTabularStorage.test.ts— all passnpx vitest run packages/test/src/test/storage-tabular/IndexedDbTabularStorage.integration.test.ts— all passnpx vitest run packages/test/src/test/storage-tabular/CachedTabularStorage.integration.test.ts— all passnpx vitest run packages/test/src/test/storage-tabular/TelemetryTabularStorage.test.ts— all passnpx vitest run packages/test/src/test/storage-tabular/SupabaseTabularStorage.integration.test.ts— all passhttps://claude.ai/code/session_01KJmJwgy4qsAK2iXXrC2yM4
Generated by Claude Code