feat(cli): rewrite db validate as eql validate for the EQL v3 domain vocabulary

.changeset/olive-poems-guess.md

@@ -0,0 +1,29 @@
+---
+'@cipherstash/stack': minor
+---
+
+Add `EncryptionClient.getSchemas()` — the tables passed to
+`Encryption({ schemas })`, returned by reference.
+
+This is the domain-bearing view of your schema. `getEncryptConfig()` returns
+what the FFI consumes: each column builds to `{ cast_as, indexes }`, and the
+concrete EQL v3 domain name is dropped. That makes `cast_as: 'number'` with an
+`ope` index ambiguous across `eql_v3_integer_ord`, `smallint_ord`, `real_ord`,
+`double_ord` and `numeric_ord` — so tooling that has to reason about the
+*declared* domain (schema linting, drift-checking a live database's
+`information_schema.columns.domain_name`) could not recover it from a client
+alone.
+
+`getSchemas()` closes that gap. Read a column's domain with
+`column.getEqlType()`, its capabilities with `column.getQueryCapabilities()`,
+and its DB name with `column.getName()`:
+
+```typescript
+for (const table of client.getSchemas()) {
+  for (const column of Object.values(table.columnBuilders)) {
+    console.log(table.tableName, column.getName(), column.getEqlType())
+  }
+}
+```
+
+`stash eql validate` is the first consumer.

.changeset/proud-ravens-repeat.md

@@ -0,0 +1,84 @@
+---
+'stash': minor
+---
+
+Rewrite `db validate` as `eql validate`, for the EQL v3 domain-type vocabulary.
+
+**Fixes a false finding on the most ordinary v3 columns.** The old rule set
+checked for `ore` / `unique` / `match` / `ste_vec` indexes and never learned
+about `ope`. EQL v3's default ordering domains emit `ope`, so
+`types.IntegerOrd('age')` and `types.TimestampOrd('created_at')` were both
+reported as "Column is encrypted but has no indexes — it will not be
+searchable". They are now silent.
+
+The command reads your tables through the new
+`EncryptionClient.getSchemas()`, so it sees each column's **concrete domain**
+rather than the lossy encrypt config, and gains a database pass when one is
+reachable.
+
+Schema checks (no database needed):
+
+| Rule | Severity |
+|---|---|
+| An `_ord_ore` domain is declared — its ORE operator class needs superuser | Warning |
+| Storage-only column: encrypts and decrypts, carries no query terms | Info |
+| Searchable `boolean` column | Error |
+| Free-text `match` index on a non-text domain | Error |
+| Encrypted-JSONB (`ste_vec`) index without `types.Json` | Error |
+
+Database checks (skipped with a notice, not a failure, when no database is
+reachable):
+
+| Rule | Severity |
+|---|---|
+| EQL v3 is not installed — reported once, remaining database checks skipped | Error |
+| A declared table lives in a different schema than the one searched | Warning |
+| A declared table is in the searched schema but invisible to the connected role | Warning |
+| A declared table name carries a schema qualifier (`schema.table`) — not checked | Warning |
+| A declared table exists in no schema at all | Error |
+| A declared column is missing from a table that was found | Error |
+| The database column's domain differs from the declared one | Error |
+| The database column is still plain (no EQL domain) | Error |
+| An `_ord_ore` domain where the EQL install could not create the ORE operator class | Error |
+| A queryable column with no functional index over its term extractor | Info |
+| A declared table name that resolved in the searched schema also exists in another one | Info |
+
+`--exclude-operator-family` is removed: it warned that an `ore` index would not
+support `ORDER BY` without operator families, and the pinned EQL v3 bundle
+self-adapts. `eql install` / `eql upgrade` had already rejected the flag;
+`validate` was its last consumer.
+
+The database pass inspects `current_schema()` only, and distinguishes four
+reasons a declared table can be missing from it, so that only the last fails
+the command. In another schema (Prisma `multiSchema`, a tenant schema): a
+Warning naming that schema. Present but invisible to the connected role: a
+Warning carrying the `GRANT SELECT` to run — `information_schema` reports only
+what the role holds a privilege on, so a missing grant is not a missing
+migration. Declared as `schema.table`: a Warning saying it was not checked,
+because validate matches table names unqualified. Absent everywhere: an Error.
+Reported once per table rather than once per column.
+
+The relation lookup that answers those questions excludes `pg_*` and
+`information_schema`. Unscoped it matched the system views named `columns`,
+`domains`, `parameters`, `routines`, `sequences`, `tables` and `triggers` — all
+ordinary application table names — so a project declaring one of them that had
+not run its migration was told the table "exists in schema information_schema",
+as a Warning, and the command exited 0 on a genuinely unapplied migration.
+
+An unqualified name found in more than one schema is now reported as an Info
+naming the relation that was actually checked (`"public"."users"`) and the
+other schemas holding that name. A bare name resolves through `search_path`, so
+`users` in both `public` and Supabase's `auth` left it ambiguous which relation
+every other finding described. Info, not Warning: it must not fail or
+de-clean an ordinary Supabase project.
+
+Two of those used to exit 1 and no longer do: a privilege-invisible table and
+a schema-qualified declaration were both reported as "does not exist in any
+schema", which sent people to re-run a migration that had already run.
+
+Against a project whose `@cipherstash/stack` predates `getSchemas()`, validate
+says so and falls back to the encrypt config, running the index-derived rules
+and skipping the domain ones.
+
+`stash db validate` keeps working as a deprecated alias, like `db install` /
+`db upgrade` / `db status`. Exits 1 on errors only.

docs/plans/cip-3366-eql-validate-v3.md

@@ -0,0 +1,196 @@
+# CIP-3366 — Rewrite `eql validate` for the EQL v3 domain-type vocabulary
+
+## Step 0 — Branch & worktree
+
+```bash
+git worktree add .claude/worktrees/toby+cip-3366-eql-validate-v3 \
+  -b toby/cip-3366-rewrite-eql-validate-for-the-eql-v3-domain-type-vocabulary
+cd .claude/worktrees/toby+cip-3366-eql-validate-v3
+pnpm install
+pnpm --filter @cipherstash/stack build   # cli resolves stack through dist/
+```
+
+## Two decisions that shape everything else
+
+### 1. `EncryptConfig` is lossy — validate must read domains, not indexes
+
+`EncryptedV3Column.build()` emits only `{ cast_as, indexes }`; the comment on
+`getEqlType()` says outright "Metadata only; not emitted by `build()`". So
+`cast_as: 'number'` + `{ ope: {} }` is ambiguous across `eql_v3_integer_ord`,
+`smallint_ord`, `real_ord`, `double_ord` and `numeric_ord`. Every new rule in the
+issue — `_ord_ore` steering, bool-not-searchable, text-only match, declared-vs-observed
+drift — needs the domain name.
+
+The tables are usually imported into the client file rather than re-exported from
+it (see the scaffold's `import { users, orders } from './db/schema'`), so
+duck-typing the module namespace is unreliable. Expose them on the client instead.
+
+**`packages/stack/src/encryption/client-v3.ts`**
+
+- Add `getSchemas(): S` to the `EncryptionClient` interface.
+- Add `getSchemas: () => schemas` to the `typed` object returned by
+  `createEncryptionClient` (`schemas` is already a parameter — ~2 lines).
+- Test in `packages/stack/__tests__/`: `getSchemas()` returns the tuple and each
+  column's `getEqlType()` round-trips.
+- Changeset: `@cipherstash/stack` minor. Update `skills/stash-encryption`.
+
+**`packages/cli/src/config/index.ts`** — add `loadEncryptSchemas(path)` beside
+`loadEncryptConfig`, reusing the same jiti load and the same
+`requireUsableEncryptConfig` placeholder guard, returning
+`{ config, schemas }`. Fall back to config-only with a warning if the client
+predates `getSchemas` (older `@cipherstash/stack` in a customer repo).
+
+### 2. Empty-string ordering is not statically checkable — scope it out
+
+The issue lists "Ordered domains reject empty strings (CHECK requires non-empty
+`ob`)". That is a value-level CHECK enforced at encrypt time; nothing in the
+schema or in `information_schema` predicts it. Leave it out and file a
+follow-up to improve the *error message* on that path instead. Say so in the PR.
+
+## Step 1 — Validation core
+
+New: `packages/cli/src/commands/eql/validate.ts`. Keep the pure core separate
+from the command shell so it is unit-testable (the current file has no tests at
+all).
+
+```ts
+interface DeclaredColumn {
+  table: string
+  column: string        // DB name, from getName()
+  eqlType: string       // 'public.eql_v3_integer_ord'
+  castAs: PlaintextKind
+  queryable: boolean
+  indexes: ColumnSchema['indexes']
+}
+
+collectDeclaredColumns(schemas): DeclaredColumn[]
+validateSchemas(cols, observed?: ObservedState): ValidationIssue[]
+```
+
+Keep the existing `ValidationIssue` / `reportIssues` shape and the
+error-exits-1 contract.
+
+### Static rules (no database)
+
+| Rule | Severity | Detection |
+|---|---|---|
+| `_ord_ore` domain declared | Warning | `eqlType.endsWith('_ord_ore')` — the ORE opclass is superuser-only; steer to the `_ord` (OPE) twin, which indexes on managed Postgres |
+| Column is not queryable | Info | `!column.isQueryable()` — successor to the v2 "no indexes" Info, now correct for `types.IntegerOrd` (see the bug below) |
+| Searchable `bool` | Error | `castAs === 'boolean' && queryable` |
+| `match` index on a non-text domain | Error | `indexes.match && castAs !== 'string'` |
+| `ste_vec` without a json domain | Error | keep from v2; retarget the message at `types.Json` |
+
+Retired: the operator-family warning, `NON_STRING_CAST_TYPES`, and the
+`--exclude-operator-family` flag (`validateInstallFlags` already hard-rejects it
+with "v3 self-adapts", and `v2-retirement.test.ts:23` asserts it is gone from
+`eql install`/`upgrade` — validate is the last consumer).
+
+The bool and text-only rules are unconstructible through `types.*` today; they
+are guards for hand-authored configs and for the drift rules below, not the
+value of this step. Note that honestly in the PR.
+
+### Database rules (when a URL resolves)
+
+Skip with a `p.log.info` notice when no URL resolves — do not fail. Reuse
+`fetchPhysicalColumns` from `packages/cli/src/commands/encrypt/lib/db-readers.ts`
+(already returns `table → column → domain_name`).
+
+| Rule | Severity |
+|---|---|
+| Declared column absent from the database | Error |
+| Observed `domain_name` ≠ declared `eqlType` (minus the `public.` prefix) | Error |
+| Observed column has no domain (plain `jsonb`/`text`) | Error |
+| `_ord_ore` declared while the ORE opclass is absent | Error (upgrades the static Warning) |
+| Queryable column with no functional index over its extractor | Info |
+
+ORE availability probe — mirrors the shipped bundle's own fallback test
+(`@cipherstash/eql@3.0.4` `dist/sql/cipherstash-encrypt.sql`, the
+`ore_fallback.sql` `DO` block):
+
+```sql
+SELECT EXISTS (
+  SELECT 1
+  FROM pg_catalog.pg_opclass c
+  JOIN pg_catalog.pg_am am ON am.oid = c.opcmethod
+  WHERE am.amname = 'btree'
+    AND c.opcdefault
+    AND c.opcintype = to_regtype('eql_v3_internal.ore_block_256')
+) AS ore_available;
+```
+
+`to_regtype` returns NULL instead of throwing when EQL is not installed, so the
+probe degrades to `false`. Detect not-installed separately and report that
+first, or the user gets "ORE unavailable" when the real answer is "run
+`stash eql install`".
+
+Functional-index rule: read `pg_index` + `pg_get_indexdef` and look for
+`eql_v3.eq_term` / `eql_v3.ord_term` / `eql_v3.ord_term_ore` /
+`eql_v3.match_term` over the column. This is the finding
+`skills/stash-indexing` already promises to resolve, so it keeps that skill's
+cross-reference honest.
+
+## Step 2 — Command wiring
+
+- `runEqlCommand` (`packages/cli/src/bin/main.ts`): add `case 'validate'`.
+- `runDbCommand`: `case 'validate'` warns via `messages.db.aliasDeprecated(STASH, 'validate')`
+  and forwards — same shape as `install` / `upgrade` / `status`. (The issue
+  assumes this alias already exists; it does not.)
+- `packages/cli/src/cli/registry.ts`: move the entry from the Database group to
+  the EQL group as `eql validate`; flags `SUPABASE_COMPAT_FLAG` +
+  `DATABASE_URL_FLAG` only.
+- `main.ts` HELP text (line 117): `db validate` → `eql validate`.
+- Delete `packages/cli/src/commands/db/validate.ts`.
+
+## Step 3 — Tests
+
+New `packages/cli/src/commands/eql/__tests__/validate.test.ts`, table-driven over
+real `encryptedTable` / `types.*` schemas rather than hand-built configs, with a
+fake observed-state map for the DB rules.
+
+Pin the regression this issue exists for:
+
+```ts
+// v2 validate reported both of these as "encrypted but has no indexes —
+// it will not be searchable": hasAnyIndex never learned about `ope`.
+const t = encryptedTable('users', {
+  age: types.IntegerOrd('age'),
+  createdAt: types.TimestampOrd('created_at'),
+})
+expect(validateSchemas(collectDeclaredColumns([t]))).toEqual([])
+```
+
+Extend `packages/cli/src/__tests__/v2-retirement.test.ts`: `eql validate` carries
+no `--exclude-operator-family`, and `db validate` is absent from the registry.
+
+## Step 4 — Docs, skills, changesets
+
+Skills ship in the `stash` tarball, so these are part of the change, not follow-up:
+
+- `skills/stash-cli/SKILL.md:430` — rewrite the rule table, move the section from
+  Database to EQL, drop the operator-family row.
+- `skills/stash-indexing/SKILL.md:17,272` — rename the "No indexes on an
+  encrypted column" finding to whatever Step 1 emits.
+- `skills/stash-postgres/SKILL.md:468` — cross-reference.
+- `skills/stash-encryption` — the new `getSchemas()` accessor.
+- `packages/cli/README.md:69,217,222`.
+- Scaffold comments naming `stash db validate`:
+  `packages/cli/src/commands/init/utils.ts:388,437,455,500` **and** the checked-in
+  fixtures `packages/cli/__fixtures__/scaffold/{generic,drizzle}.generated.ts` —
+  `placeholder-client-fixture.test.ts` compares them, so both move together.
+- Changesets: `@cipherstash/stack` minor (accessor), `stash` minor (command move,
+  new rules, flag removal).
+
+No wizard change needed: `ALLOWED_DLX_TOOLS` / `ALLOWED_BASH_COMMANDS` already
+allow the `stash eql` prefix.
+
+## Verification
+
+```bash
+pnpm run code:fix
+pnpm --filter @cipherstash/stack build && pnpm --filter @cipherstash/stack test
+pnpm --filter stash build && pnpm --filter stash test
+node packages/cli/dist/bin/stash.js manifest --json   # diff against skills/stash-cli
+```
+
+Manual against a live database: one `_ord_ore` column on a non-superuser role
+(Error), one drifted domain, one queryable column with no functional index.

packages/cli/README.md

@@ -66,7 +66,7 @@ export default defineConfig({
 
 The CLI loads `.env` files automatically before reading the config, so `process.env` references work without extra setup. The config file is resolved by walking up from the current working directory.
 
-Commands that consume `stash.config.ts`: `eql install`, `eql upgrade`, `db validate`, `eql status`, `db test-connection`, `schema build`, and `encrypt *`.
+Commands that consume `stash.config.ts`: `eql install`, `eql upgrade`, `eql validate`, `eql status`, `db test-connection`, `schema build`, and `encrypt *`.
 
 ---
 
@@ -214,23 +214,41 @@ The install SQL is idempotent and safe to re-run. If EQL is not installed, the c
 
 ---
 
-### `npx stash db validate`
+### `npx stash eql validate`
 
-Validate your encryption schema for common misconfigurations.
+Validate your encryption schema against the EQL v3 domain vocabulary, and — when
+a database is reachable — against what that database actually has.
 
 ```bash
-npx stash db validate [--supabase] [--exclude-operator-family]
+npx stash eql validate [--supabase] [--database-url <url>]
 ```
 
+Schema checks (no database needed):
+
+| Rule | Severity |
+|------|----------|
+| An `_ord_ore` domain, whose ORE operator class only a superuser can create | Warning |
+| Storage-only column — encrypts and decrypts, carries no query terms | Info |
+| Searchable `boolean` column | Error |
+| Free-text `match` on a non-text domain | Error |
+| Encrypted-JSONB search without `types.Json` | Error |
+
+Database checks (skipped with a notice when no database is reachable):
+
 | Rule | Severity |
 |------|----------|
-| `freeTextSearch` on a non-string column | Warning |
-| `orderAndRange` without operator families | Warning |
-| No indexes on an encrypted column | Info |
-| `searchableJson` without `dataType("json")` | Error |
+| EQL v3 is not installed (reported once; the other database checks are skipped) | Error |
+| Declared column or table absent from the database | Error |
+| The database column's domain has drifted from the declaration | Error |
+| The column is still plain (no EQL domain) | Error |
+| An `_ord_ore` domain on a database whose EQL install could not create the ORE operator class | Error |
+| Queryable column with no functional index over its term extractor | Info |
+| A declared table name that resolved in the searched schema also exists in another one | Info |
 
 The command exits with code 1 on errors (not on warnings or info).
 
+`stash db validate` still works as a deprecated alias.
+
 ---
 
 ### `npx stash db migrate`

packages/cli/__fixtures__/scaffold/drizzle.generated.ts

@@ -8,7 +8,7 @@
  * to reference the encrypted tables you declared there.
  *
  * Until that happens, the encryption client is initialised with a single
- * placeholder table so that this file compiles, and `stash db validate` and
+ * placeholder table so that this file compiles, and `stash eql validate` and
  * `stash encrypt backfill` refuse to run and point back here. (`stash
  * encrypt drop` resolves against the database and never reads this file.)
  *
@@ -57,7 +57,7 @@ import { Encryption, encryptedTable, types } from '@cipherstash/stack/v3'
 
 // REPLACE THIS. It exists only so this file compiles before you have declared
 // any encrypted tables — `Encryption` requires at least one. Swap it for your
-// real tables (see the patterns above); `stash db validate` and `stash
+// real tables (see the patterns above); `stash eql validate` and `stash
 // encrypt backfill` refuse to run while the placeholder is still here.
 export const placeholderTable = encryptedTable('__stash_placeholder__', {
   replace_me: types.Text('replace_me'),