Skip to content

refactor(stack): EQL v3 types namespace on @cipherstash/stack/eql/v3#541

Open
tobyhede wants to merge 10 commits into
feat/eql-v3-text-search-schemafrom
feat/eql-v3-types-module
Open

refactor(stack): EQL v3 types namespace on @cipherstash/stack/eql/v3#541
tobyhede wants to merge 10 commits into
feat/eql-v3-text-search-schemafrom
feat/eql-v3-types-module

Conversation

@tobyhede

@tobyhede tobyhede commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Refactors the EQL v3 authoring surface: the per-domain encrypted<Domain>Column factories become a single types namespace whose members mirror the underlying eql_v3.<name> domains, on the renamed @cipherstash/stack/eql/v3 subpath.

import { encryptedTable, types } from "@cipherstash/stack/eql/v3";

const events = encryptedTable("events", {
  actor:     types.TextEq("actor"),           // equality
  weight:    types.Int4Ord("weight"),         // order + range
  createdAt: types.Timestamptz("created_at"), // storage only
});

Before → after — the name maps 1:1 to the EQL v3 domain:

// before                                          // after
encryptedTextEqColumn("actor")                     types.TextEq("actor")          // eql_v3.text_eq
encryptedInt4OrdColumn("weight")                    types.Int4Ord("weight")        // eql_v3.int4_ord
encryptedTimestamptzColumn("created_at")            types.Timestamptz("created_at")// eql_v3.timestamptz
encryptedTextSearchColumn("email").freeTextSearch() types.TextSearch("email").freeTextSearch()

Per-domain plaintext inference and compile-time queryability are unchanged:

import { Encryption } from "@cipherstash/stack";
import type { InferPlaintext } from "@cipherstash/stack/eql/v3";

type Events = InferPlaintext<typeof events>;
// { actor: string; weight: number; createdAt: Date }

const client = await Encryption({ schemas: [events] });

await client.encryptQuery(30, { table: events, column: events.weight, queryType: "orderAndRange" });
await client.encryptQuery(new Date(), { table: events, column: events.createdAt });
//                                                            ^ type error: storage-only, not queryable

The @cipherstash/stack/v3 typed client re-exports types (in place of the standalone builders):

import { EncryptionV3, encryptedTable, types } from "@cipherstash/stack/v3";

const users = encryptedTable("users", { email: types.TextSearch("email") });
const client = await EncryptionV3({ schemas: [users] });

await client.encrypt("a@b.com", { table: users, column: users.email }); // ok
await client.encrypt(123,       { table: users, column: users.email }); // ✗ number ≠ string

types members

One member per generated EQL v3 domain (PascalCase of the eql_v3.<name>):

Int4 Int4Eq Int4OrdOre Int4Ord · Int2* · Date* · Timestamptz* · Numeric* · Text TextEq TextMatch TextOrdOre TextOrd TextSearch · Bool · Float4* Float8* (int8/bigint still omitted pending lossless FFI I/O). Each returns its concrete branded class, so per-column inference stays precise.

Scope

  • Splits the 992-line schema/v3/index.ts into src/eql/v3/{columns,types,table,index}.ts.
  • Renames the subpath schema/v3eql/v3 (exports, tsup entry, FTA gate, ./v3 re-export); the old subpath and the standalone factories are removed.
  • Behaviour preserved: same classes, nominal typing, and build() output (text_search stays byte-identical).

Verified: schema/v3 no longer resolves (ERR_PACKAGE_PATH_NOT_EXPORTED) and factories are undefined on both subpaths; 101 v3 runtime + 50 type tests pass; FTA scores all 4 files (max 68.68 < 72). Stacked on #535.

@tobyhede tobyhede requested a review from a team as a code owner July 3, 2026 01:13
@changeset-bot

changeset-bot Bot commented Jul 3, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 430392f

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes changesets to release 6 packages
Name Type
@cipherstash/stack Minor
@cipherstash/bench Patch
@cipherstash/prisma-next Patch
@cipherstash/basic-example Patch
@cipherstash/prisma-next-example Patch
@cipherstash/e2e Patch

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Jul 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d50ff535-fba1-4812-8bdb-b01dcbcae82f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/eql-v3-types-module

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

tobyhede added 6 commits July 3, 2026 11:55
Replace the 35 verbose `encrypted<Domain>Column` factories with a single
`types` namespace whose members mirror the underlying `eql_v3.<name>` domains
1:1 (`types.TextEq`, `types.Int4Ord`, `types.Timestamptz`, …), and split the
992-line `src/schema/v3/index.ts` into a cohesive module under `src/eql/v3/`
(`columns.ts`, `types.ts`, `table.ts`, curated `index.ts`).

The authoring subpath is renamed `@cipherstash/stack/schema/v3` ->
`@cipherstash/stack/eql/v3`; the `./v3` typed-client surface now re-exports the
`types` namespace instead of the standalone factories. Behaviour is preserved:
same classes, same nominal-typing mechanism, same `build()` output.

- Rewire tsup entry, package.json exports/typesVersions/analyze:complexity,
  the `@/eql/v3` re-export, `[eql/v3]` error prefix, and fta-v3.yml paths.
- Migrate all v3 tests + CJS smoke test to `types.*` and the new subpath.
- Reconcile the three unreleased changesets and refresh tracked v3 design docs
  (supersede banners on completed-work records; correct the not-yet-built
  Stryker gate spec's single-file premise for the 4-file split).

Verified: build emits dist/eql/v3; schema/v3 subpath and factories are gone
(ERR_PACKAGE_PATH_NOT_EXPORTED, undefined on both subpaths); 101 v3 runtime +
50 type tests pass; FTA scores all 4 files (max 68.68 < 72); e2e authoring
config byte-matches expected.
Two JS properties whose builders resolve to the same DB name (getName())
silently overwrote in the built config — the later column won and the first's
config was dropped. Throw instead, matching the existing duplicate-tableName
guard in buildEncryptConfig and the reserved-key guard in encryptedTable.

Regression tests: `EncryptedTable.build()` and `buildEncryptConfig` both throw
on a duplicate DB name (schema-v3.test.ts, eql_v3 encryptedTable block).
The structural builder contracts (BuildableColumn, BuildableQueryColumn,
BuildableV3QueryableColumn, BuildableTable, BuildableTableColumns) and the
encryptModel/bulkEncryptModels return-type mapper (EncryptedFromBuildableTable)
appear in public return positions but were not re-exported from
`@cipherstash/stack/types`, so consumers could not name them — an inconsistency
with the already-exposed `EncryptedFromSchema`. No build breakage (the mapped
types were emitted inline); this closes the nameability gap.

Regression guard: types-public-surface.test-d.ts imports each contract from the
public `@/types-public` entrypoint (a missing re-export fails typecheck).

Note: these types are inherited from the base branch (feat/eql-v3-text-search-schema,
PR #535); the export is added here in response to review feedback on the stacked PR.
The v3-matrix domain suite (catalog.ts + matrix tests) landed on the base
branch via PR #540 after this branch was cut, and used the pre-refactor
`@/schema/v3` path and `encrypted<Domain>Column` factories. Retarget it to
`@/eql/v3` and the `types.*` namespace so the base's matrix coverage keeps
working on top of the refactor. `EqlTypeForColumn` (which #540's catalog.ts
consumes) is preserved — ported into eql/v3/columns.ts and re-exported from the
barrel during the rebase.

Post-rebase reconciliation only; no behavior change.
@tobyhede tobyhede force-pushed the feat/eql-v3-types-module branch from b98ad89 to 2a078b8 Compare July 3, 2026 02:02
tobyhede added 2 commits July 3, 2026 13:37
Close two coverage gaps on the eql/v3 branch that only live/e2e tests
touched:

- encrypt-lock-context-guards: assert NaN/+Inf/-Inf are rejected on the
  `encrypt(...).withLockContext(...)` path and short-circuit before the
  FFI call. The non-lock guards run only under the live number-protect
  suite; the lock-context arm (encrypt.ts:163-168) had no coverage.
- wasm-inline-new-client: assert the protect-ffi 0.25 single-object
  `newClient({ strategy, encryptConfig, clientId, clientKey })` shape,
  incl. cast_as normalisation. Previously exercised only by the
  secret-gated Deno e2e, so a regression to the 0.24 two-arg form would
  pass normal CI.

Both run offline (mocked FFI).
The all-35-domain live Postgres suite was force-skipped (describe.skip, not
credential-gated) after `beforeAll`'s dynamic INSERT crashed with
`invalid input syntax for type json` (PR #540). That crash was a postgres.js
serialization gap — a bare ciphertext object stringified to "[object Object]" —
and was fixed 32 minutes later by wrapping every INSERT param in `sql.json(...)`
(commit 53cf854). The force-skip was simply left stale; it is not an FFI
limitation.

Restore the credential-gated form (`LIVE_EQL_V3_PG_ENABLED ? describe :
describe.skip`) as the file's own comment instructed, so the 35-domain SQL
round-trip runs in CI (which supplies DATABASE_URL + CS_* creds) and self-skips
locally. The genuine FFI-level skip — timestamptz `cast_as:'date'` time-of-day
truncation in schema-v3-client.test.ts — stays skipped (needs a native
'timestamp' cast_as variant). timestamptz matrix cases are unaffected (midnight
samples, no truncation).
@tobyhede tobyhede requested review from coderdan and freshtonic July 3, 2026 04:07

@freshtonic freshtonic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve but I notice that timestamptz is back - I thought we finally reached a conclusion (that Claude's advice was incorrect)?

…equirement)

The `eql_v3.text_ord` and `eql_v3.text_ord_ore` Postgres domains require BOTH
`hm` (HMAC) and `ob` (ORE) in the stored ciphertext — text equality is
HMAC-based (their `eql_v3.eq_term` extracts `hm`), unlike numeric/date order
domains which answer equality via `ob` and need only ORE. The SDK's
`indexesForCapabilities` treated every order/range domain identically, emitting
`ore` only, so text-order ciphertexts lacked `hm` and a real INSERT failed with
`value for domain eql_v3.text_ord_ore violates check constraint`. (Surfaced by
re-enabling matrix-live-pg; masked before by the suite skip.)

Make index derivation castAs-aware: emit `unique` (hm) when equality is
answered via HMAC — equality-only domains of any type, AND text order domains
(`string` + order/range). Numeric/date order domains are unchanged (`ore` only).

Query path follows automatically: `resolvesEqualityViaOre` only fires when
`unique` is absent, so text-order equality now resolves to the `hm` index
(eq_term) while numeric/date order equality still resolves to `ore`.

TDD: text_ord/text_ord_ore build() now emits { unique, ore }; numeric order
stays { ore }; text-order equality resolves to unique. Catalog + matrix build()
assertions updated (TEXT_ORD_IDX). Verified against the eql_v3 domain checks in
the fixture; live SQL runs in CI.

@coderdan coderdan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tres excellente!

Comment on lines +2 to +9
BOOL,
DATE,
DATE_EQ,
DATE_ORD,
DATE_ORD_ORE,
EncryptedBoolColumn,
EncryptedDateColumn,
EncryptedDateEqColumn,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between BOOL and EncryptedBoolColumn?

import * as ffi from '@cipherstash/protect-ffi'

const users = encryptedTable('users', {
score: encryptedColumn('score').dataType('number').equality().orderAndRange(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIU, the domain types will basically get mapped into the same thing this fluent-builder does. But the test might be less brittle if we used a domain type here directly?


beforeEach(async () => {
vi.clearAllMocks()
process.env.CS_WORKSPACE_CRN = 'crn:ap-southeast-2.aws:test-workspace'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this hardcoded?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh - because its not a real workspace ID. Carry on!

@@ -0,0 +1,91 @@
/**
* Offline guard tests for the lock-context encrypt path.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this also address #530?

Comment on lines +27 to 28
.TextSearch('email')
.freeTextSearch({

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty great tbh! We should make sure its documented in the Typedoc.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The merge semantics are documented in the JSDoc on freeTextSearch() (columns.ts:458-462): each provided key replaces its default, omitted keys keep the default, mirroring v2's opts?.x ?? default. There's currently no Typedoc pipeline in the repo — happy to set one up as a follow-up if that's what you're after.

coderdan commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

From test-gap-review:

Test-coverage review — eql/v3 authoring DSL

Verdict: coverage is strong; one small lopsided gap worth closing.

The bulk of this PR is a rename/refactor (schema/v3eql/v3, the encrypted*Column factories → the types.* namespace) whose behavior is pinned by the exhaustive type-driven V3_MATRIX: matrix.test.ts asserts builder/getName/getEqlType/getQueryCapabilities/isQueryable/build() for all 35 domains, matrix-live*.test.ts roundtrips every domain through live FFI + Postgres, and the genuinely-new logic (text-order domains carrying the hm/unique index) already has dedicated regression tests. New public Buildable* type exports are guarded by types-public-surface.test-d.ts. The table guards (duplicate DB-column name, duplicate table name, reserved-key collision incl. prototype members) are all tested.

Inline gap (1)

packages/stack/src/eql/v3/columns.ts:355 — base-class match-index clone in indexesForCapabilities() (used by eql_v3.text_match) has no cross-column mutable-state test, while the parallel EncryptedTextSearchColumn.build() override does. Classic lopsided-clone-path anti-pattern: a regression to a shared const default would be caught for text_search but slip through for text_match. Sketch mirrors the existing "built columns share no mutable state" test using types.TextMatch(...). Passes today; fails if the base path stops cloning.

Additional gaps (body only)

  • freeTextSearch() called with no argument (opts === undefined) — untested branch; every test passes an explicit opts object.
  • buildEncryptConfig() with zero tables — untested empty-input boundary. Low value.

No security issues noticed incidentally.

… guards

Test-only additions (separated from the in-flight EQL v3 bundle upgrade so they
land on this branch, not the bundle branch):

- encrypt-lock-context-guards.test.ts: run every non-finite-number guard case
  against BOTH a v2 fluent-builder column and a v3 domain column, since the
  guard lives on the shared EncryptOperationWithLockContext.
- schema-v3.test.ts: `.freeTextSearch()` no-arg is a no-op (pins the
  opts===undefined branch); a text_match mutable-state aliasing guard (base-class
  match-clone path, which the text_search-only test can't cover); and
  buildEncryptConfig() with zero tables yields { v: 1, tables: {} }.
- wasm-inline-strategy.test.ts: Biome line-wrap formatting only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants