diff --git a/.claude/skills/driver-test-runner/SKILL.md b/.claude/skills/driver-test-runner/SKILL.md index 0b4aa37de9..df8795bcc9 100644 --- a/.claude/skills/driver-test-runner/SKILL.md +++ b/.claude/skills/driver-test-runner/SKILL.md @@ -24,18 +24,12 @@ The skill accepts optional arguments: The driver suite runs over a runtime × SQLite-backend × encoding matrix defined in `rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.ts`. The runtime dimension has two values: -- **`native`** — NAPI bindings (`@rivetkit/rivetkit-napi`). Pairs with `sqlite=local` (the in-process SQLite VFS). This is the default when no env override is set. -- **`wasm`** — WebAssembly bindings (`@rivetkit/rivetkit-wasm`). Wasm **cannot** use local SQLite; it must pair with `sqlite=remote` (executes SQL through the engine over the wire). Setting `RIVETKIT_DRIVER_TEST_RUNTIME=wasm` with `RIVETKIT_DRIVER_TEST_SQLITE=local` fails fast. +- **`native`** — NAPI bindings (`@rivetkit/rivetkit-napi`). Pairs with `sqlite=local` for the primary native driver pass. +- **`wasm`** — WebAssembly bindings (`@rivetkit/rivetkit-wasm`). Wasm **cannot** use local SQLite; it must pair with `sqlite=remote` (executes SQL through the engine over the wire). The skill defaults to running each test file twice: once on `native/local` and once on `wasm/remote`, each at `encoding=bare`. A file is checked off only when both runtimes pass. -Env overrides recognized by the test harness: - -- `RIVETKIT_DRIVER_TEST_RUNTIME` — comma-separated subset of `native,wasm`. -- `RIVETKIT_DRIVER_TEST_SQLITE` — comma-separated subset of `local,remote`. -- `RIVETKIT_DRIVER_TEST_ENCODING` — comma-separated subset of `bare,cbor,json`. - -When **any** of these env vars is set, the inner describe block name changes from `encoding ()` to `runtime () / sqlite () / encoding ()`. The skill always sets the env vars, so the longer form is always what `-t` must match. +The test harness does not read environment variables for matrix selection. Always select matrix cells with Vitest `-t` using the full inner describe name: `runtime () / sqlite () / encoding ()`. ## How It Works @@ -175,21 +169,20 @@ For each unchecked row in order, run the runtimes selected by the `runtime` arg **b) Build the filter command:** -Each suite lives in its own file under `rivetkit-typescript/packages/rivetkit/tests/driver/.test.ts`. With env overrides set, the describe block nesting is: +Each suite lives in its own file under `rivetkit-typescript/packages/rivetkit/tests/driver/.test.ts`. The describe block nesting is: ``` > static registry > runtime () / sqlite () / encoding () > ``` +Always use Vitest `-t` for driver matrix cells. Include runtime, SQLite backend, and encoding in the pattern so a partial match does not accidentally select another matrix cell. + Base command (native): ```bash cd rivetkit-typescript/packages/rivetkit && \ - RIVETKIT_DRIVER_TEST_RUNTIME=native \ - RIVETKIT_DRIVER_TEST_SQLITE=local \ - RIVETKIT_DRIVER_TEST_ENCODING=bare \ pnpm test tests/driver/.test.ts \ - -t "static registry.*runtime \\(native\\) / sqlite \\(local\\) / encoding \\(bare\\).*" \ + -t "runtime \(native\) / sqlite \(local\) / encoding \(bare\)" \ > /tmp/driver-test-current.log 2>&1 echo "EXIT: $?" ``` @@ -198,11 +191,8 @@ Base command (wasm): ```bash cd rivetkit-typescript/packages/rivetkit && \ - RIVETKIT_DRIVER_TEST_RUNTIME=wasm \ - RIVETKIT_DRIVER_TEST_SQLITE=remote \ - RIVETKIT_DRIVER_TEST_ENCODING=bare \ pnpm test tests/driver/.test.ts \ - -t "static registry.*runtime \\(wasm\\) / sqlite \\(remote\\) / encoding \\(bare\\).*" \ + -t "runtime \(wasm\) / sqlite \(remote\) / encoding \(bare\)" \ > /tmp/driver-test-current.log 2>&1 echo "EXIT: $?" ``` @@ -275,7 +265,7 @@ If both runtime boxes are now checked, the file is fully done; advance to the ne **e) If tests fail:** 1. Do NOT move to the next runtime or file. -2. Narrow down to the first failing test using a more specific `-t` filter (keep the same env vars). +2. Narrow down to the first failing test by adding enough test-name text to the same `-t` pattern. 3. Read the error output to understand the failure. 4. Append to the log: @@ -292,18 +282,16 @@ If both runtime boxes are now checked, the file is fully done; advance to the ne ### 5. Narrowing scope on failure -If a file group fails, narrow to individual tests while keeping the same runtime env vars: +If a file group fails, keep the full matrix selector and append enough test-name text to isolate the failing test: ```bash cd rivetkit-typescript/packages/rivetkit && \ - RIVETKIT_DRIVER_TEST_RUNTIME= \ - RIVETKIT_DRIVER_TEST_SQLITE= \ - RIVETKIT_DRIVER_TEST_ENCODING=bare \ pnpm test tests/driver/.test.ts \ - -t "static registry.*runtime \\(\\) / sqlite \\(\\) / encoding \\(bare\\).*.*" \ + -t "runtime \(\) / sqlite \(\) / encoding \(bare\).*" \ > /tmp/driver-test-narrow.log 2>&1 ``` +Do not use `-t` as a flake workaround. It is only for selecting the intended matrix cell and, when needed, a specific failing test. If the bug only appears on one runtime, that's a strong signal — focus the diff hunt on the corresponding runtime adapter (`napi-runtime.ts` / `wasm-runtime.ts`) and any wasm-feature-gated code in `rivetkit-core` and `rivetkit-typescript/packages/rivetkit-wasm`. ### 6. Completion @@ -327,6 +315,6 @@ Report summary: 3. **Fix before advancing.** Do not skip a failing runtime/file to test the next one (unless the user says to skip). 4. **Always pipe to file.** Never rely on inline terminal output for test results. Always write to `/tmp/driver-test-current.log` and grep afterward. 5. **Track everything.** Every run gets logged in the progress file with its runtime tag. -6. **Always set the env vars.** Even when running a single runtime, set `RIVETKIT_DRIVER_TEST_RUNTIME`, `RIVETKIT_DRIVER_TEST_SQLITE`, and `RIVETKIT_DRIVER_TEST_ENCODING`. The describe path depends on having any of them set. +6. **Always use `-t` for matrix selection.** Include runtime, SQLite backend, and encoding in the selector. Do not scope the driver matrix with env vars. 7. **Never pair `wasm` with `local` SQLite.** The harness throws on this combination. If a wasm run somehow needs local SQLite to repro a bug, that's a bug in the matrix, not a workaround to apply. 8. **Respect timeouts.** Set a 600-second timeout for slow tests (sleep, lifecycle, stress). Use 120 seconds for fast tests. Wasm runs may be slower than native — extend timeouts proportionally if you see consistent timeouts on wasm only. diff --git a/rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.test.ts b/rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.test.ts index b97edf88b1..e430ca232d 100644 --- a/rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.test.ts +++ b/rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.test.ts @@ -1,38 +1,10 @@ -import { afterEach, describe, expect, test } from "vitest"; +import { describe, expect, test } from "vitest"; import { getDriverMatrixCells, SQLITE_DRIVER_MATRIX_OPTIONS, } from "./shared-matrix"; -const previousRuntimeEnv = process.env.RIVETKIT_DRIVER_TEST_RUNTIME; -const previousSqliteEnv = process.env.RIVETKIT_DRIVER_TEST_SQLITE; -const previousEncodingEnv = process.env.RIVETKIT_DRIVER_TEST_ENCODING; - -function restoreMatrixEnv() { - if (previousRuntimeEnv === undefined) { - delete process.env.RIVETKIT_DRIVER_TEST_RUNTIME; - } else { - process.env.RIVETKIT_DRIVER_TEST_RUNTIME = previousRuntimeEnv; - } - - if (previousSqliteEnv === undefined) { - delete process.env.RIVETKIT_DRIVER_TEST_SQLITE; - } else { - process.env.RIVETKIT_DRIVER_TEST_SQLITE = previousSqliteEnv; - } - - if (previousEncodingEnv === undefined) { - delete process.env.RIVETKIT_DRIVER_TEST_ENCODING; - } else { - process.env.RIVETKIT_DRIVER_TEST_ENCODING = previousEncodingEnv; - } -} - describe("driver matrix cells", () => { - afterEach(() => { - restoreMatrixEnv(); - }); - test("excludes wasm with local SQLite from the normal matrix", () => { const cells = getDriverMatrixCells(SQLITE_DRIVER_MATRIX_OPTIONS); @@ -73,12 +45,18 @@ describe("driver matrix cells", () => { ]); }); - test("fails fast when env explicitly selects wasm with local SQLite", () => { - process.env.RIVETKIT_DRIVER_TEST_RUNTIME = "wasm"; - process.env.RIVETKIT_DRIVER_TEST_SQLITE = "local"; + test("defaults to both runnable runtime pairs", () => { + const cells = getDriverMatrixCells({ encodings: ["bare"] }); - expect(() => getDriverMatrixCells()).toThrow( - /WebAssembly runtime cannot use local SQLite/, - ); + expect( + cells.map( + (cell) => + `${cell.runtime}/${cell.sqliteBackend}/${cell.encoding}`, + ), + ).toEqual([ + "native/local/bare", + "native/remote/bare", + "wasm/remote/bare", + ]); }); }); diff --git a/rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.ts b/rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.ts index 7a41cc66fc..1ad5e0b016 100644 --- a/rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.ts +++ b/rivetkit-typescript/packages/rivetkit/tests/driver/shared-matrix.ts @@ -42,66 +42,14 @@ export interface DriverMatrixCell { skipReason?: string; } -function envList( - name: string, - allowed: readonly T[], -): T[] | undefined { - const value = process.env[name]; - if (!value) { - return undefined; - } - - const values = value - .split(",") - .map((part) => part.trim()) - .filter(Boolean); - for (const item of values) { - if (!allowed.includes(item as T)) { - throw new Error( - `invalid ${name} value '${item}', expected one of ${allowed.join(", ")}`, - ); - } - } - return values as T[]; -} - -function hasEnvMatrixOverride() { - return ( - process.env.RIVETKIT_DRIVER_TEST_RUNTIME !== undefined || - process.env.RIVETKIT_DRIVER_TEST_SQLITE !== undefined || - process.env.RIVETKIT_DRIVER_TEST_ENCODING !== undefined - ); -} - export function getDriverMatrixCells( options: DriverMatrixOptions = {}, ): DriverMatrixCell[] { - const envEncodings = envList("RIVETKIT_DRIVER_TEST_ENCODING", [ - "bare", - "cbor", - "json", - ] as const); - const envRuntimes = envList("RIVETKIT_DRIVER_TEST_RUNTIME", [ - "native", - "wasm", - ] as const); - const envSqliteBackends = envList("RIVETKIT_DRIVER_TEST_SQLITE", [ - "local", - "remote", - ] as const); - const encodings = envEncodings ?? - options.encodings ?? ["bare", "cbor", "json"]; - const runtimes = envRuntimes ?? options.runtimes ?? ["native"]; - const sqliteBackends = envSqliteBackends ?? - options.sqliteBackends ?? ["local"]; + const encodings = options.encodings ?? ["bare", "cbor", "json"]; + const runtimes = options.runtimes ?? ["native", "wasm"]; + const sqliteBackends = options.sqliteBackends ?? ["local", "remote"]; const cells: DriverMatrixCell[] = []; - if (envRuntimes?.includes("wasm") && envSqliteBackends?.includes("local")) { - throw new Error( - "invalid driver test matrix: WebAssembly runtime cannot use local SQLite. Set RIVETKIT_DRIVER_TEST_SQLITE=remote for wasm driver tests.", - ); - } - for (const runtime of runtimes) { for (const sqliteBackend of sqliteBackends) { if (runtime === "wasm" && sqliteBackend === "local") { @@ -133,10 +81,6 @@ export function describeDriverMatrix( registryVariantNames.has(variant.name), ); const cells = getDriverMatrixCells(options); - const includeSqliteDimensions = - hasEnvMatrixOverride() || - options.runtimes !== undefined || - options.sqliteBackends !== undefined; describeDriverSuite(suiteName, () => { for (const variant of variants) { @@ -151,9 +95,7 @@ export function describeDriverMatrix( }); for (const cell of cells) { - const suite = includeSqliteDimensions - ? `runtime (${cell.runtime}) / sqlite (${cell.sqliteBackend}) / encoding (${cell.encoding})` - : `encoding (${cell.encoding})`; + const suite = `runtime (${cell.runtime}) / sqlite (${cell.sqliteBackend}) / encoding (${cell.encoding})`; if (cell.skipReason) { describe.skip(`${suite}: ${cell.skipReason}`, () => {});