fix(metadata-protocol): refuse a dotted projection instead of widening the response to every field (#7532)

[claude]

Fixes #7532

What was wrong

POST /api/v1/data/showcase_invoice/query {"fields":["name","account.name"]} answered 200 carrying every business field — strictly more data than was asked for — and no resolved account.name. GET ...?$select=name,account.name did the same.

assertProjectionFieldsExist validated only f.split('.')[0], so a dotted entry cleared the #4226 unknown-name gate on its head segment (account really is a field) and travelled on to the driver as a projection column.

Where the widening actually happens

The card attributed the fallback to the ingress gate. It is one layer lower, and I measured both:

• The protocol does not drop the name. Driven through protocol.findData against a stub driver, fields:['title','project_id.name'] reaches the driver as ["title","project_id.name"] — forwarded verbatim.
• SqlDriver widens it. Driven against a real SqlDriver (better-sqlite3, :memory:), calling driver.find directly:

no projection                  -> account amount created_at id name status updated_at
fields ['name']                -> name                        <- a plain name narrows
fields ['name','account.name'] -> account amount created_at id name status updated_at
fields ['account.name']        -> account amount created_at id name status updated_at

The dotted rows are byte-identical to no projection at all. Knex renders "account"."name" against a table that was never joined, sqlite answers no such column, and the #3821 recovery ladder retries select('*') because rows matter more than the projection.

Per ruling, this PR closes the ingress gate and leaves the driver ladder alone; packages/driver-sql is untouched. The driver behaviour is filed separately as a finding: #7589, carrying this measurement and a drop-in reproduction harness. It is defence-in-depth behind this gate rather than a live user-facing defect once this lands — but it still governs every caller that reaches SqlDriver without passing this ingress (hooks, flows, reports, expand sub-reads, registry-less hosts).

What this does

A dotted entry on the projection axis is now 400 INVALID_FIELD, ordered unknown > dotted so this axis reports the same complaint first as assertSortFieldsExist already does. Two messages, because the fixes differ:

• head is a reference field → names the relationship crossed, prescribes expand (or denormalising the value onto the queried object).
• head is a real but non-reference column → says a dotted path reaches only whole columns, and to read into the value in the caller. expand would be the wrong prescription here.

This also settles the card's second complaint: an unknown plain column was a 400 while an unknown dotted one was a 200 with every field — one mistake, opposite verdicts on one endpoint, depending on spelling.

Precedent. #5918 faced this exact shape on the analytics measures axis and ruled the same way — refuse the dotted member loudly, naming the caller's original spelling, because there is no correct answer to converge on. That is also the distinction from #5739, where refusing would have rejected queries that already compiled correctly.

Is expand the sanctioned mechanism for related data on this door?

Yes — and after this PR it is the only one. Confirmed against query-syntax.mdx §4 and the spec: expand resolves reference fields into the related record via batched $in reads, driver-agnostic, and its nested QueryAST can both filter (where) and select (fields) the related record's columns. It is what the refusal points at, and a GUARD test pins that it still delivers.

One sharp edge, measured while writing that test and pinned: the projection must retain the foreign-key column. fields:['title'], expand:'project_id' projects the FK away and expansion has nothing to resolve; fields:['title','project_id'] works. Worth knowing since this PR now routes callers here.

⚠️ Follow-up this PR deliberately does not do

Several in-repo surfaces still offer a dotted fields path as the way to read one related column. Every one of them describes behaviour no driver implements — they were already wrong before this PR, and they are now also contradicted by a 400:

surface	location
spec fields description (normative, generates the JSON Schema + contract docs)	packages/spec/src/data/query.zod.ts:484
type pins	packages/spec/src/recursive-schema-input-assertions.ts:74,99
query.joins retirement prescription	packages/spec/src/migrations/entries/semantic/17.query-joins-retired.ts:10
nested-select retirement prescription	.../17.query-field-node-object-form-retired.ts:8
shipped JSON Schema artifacts	packages/spec/json-schema/api/{FindDataRequest,ExportRequest}.json, objectstack.json
docs	content/docs/protocol/objectql/query-syntax.mdx:140,972; content/docs/references/data/query.mdx:125,134; content/docs/references/api/contract.mdx:407

Aligning these is cross-package spec/docs surface that regenerates artifacts, and doing it inside this PR while #7534 is in flight is exactly the collision the serialization constraint exists to prevent.

Filed as #7601 — carrying this table (normative surface first), the measurement that every one of them describes behaviour no driver implements, the coherent-split note (the zod schema still parses a dotted string because it is a shape check — the fix is the .describe() prose and the generated artifacts, not the schema type), and the expand foreign-key edge below so the corrected prose carries it. Ruled a restore-invariant fix (declared = enforced), not an open product question; scheduling is what kept it out of this PR.

Note the zod schema still parses a dotted string (it is a shape check, z.string()); the semantic refusal is this ingress gate. That split is coherent, but the .describe() prose is now misleading.

File surface

file	change
packages/metadata-protocol/src/protocol.ts	assertProjectionFieldsExist gains the dotted verdict after the unknown one; shape-rejection hint no longer prescribes select=owner.name; docblock records the measurement
packages/objectql/src/query-expression-conformance.test.ts	inverts the test that pinned the old behaviour; adds 11 tests (4 GUARD)
.changeset/dotted-projection-refused.md	patch, @objectstack/metadata-protocol

resolveQueryFields is unchanged — the projection path only calls into it. packages/driver-sql is untouched.

The inverted test

query-expression-conformance.test.ts carried it('a dotted path is still accepted — the replacement the rejection prescribes'), asserting findData({select:'parent_id.title'}) resolves, on the reasoning that "the head segment is validated here, the tail resolved downstream". The tail is resolved nowhere — the measurement above is what that test was actually protecting. It now asserts the refusal and records why it flipped.

Reverse verification

Predictions written before running. Source reverted via patch file (never git stash), new tests retained.

test	predicted	on revert
dotted refused — POST /query body fields	RED	RED	✅
dotted refused — GET ?$select=	RED	RED	✅
dotted refused — GET ?select=	RED	RED	✅
dotted refused — fields comma string	RED	RED	✅
projection that is ONLY a dotted path	RED	RED	✅
message names relation + prescribes expand	RED	RED	✅
non-relation dotted head gets other wording	RED	RED	✅
inverted conformance test	RED	RED	✅
GUARD plain projection narrows to exact key set	green both	green	✅
GUARD same control through the GET door	green both	green	✅
GUARD unknown plain column still 400 (#4226)	green both	green	✅
GUARD unknown HEAD still the unknown verdict (precedence)	green both	green	✅
GUARD expand still delivers related record	green both	green	✅

8 failed | 122 passed on revert — exactly the 8 predicted, no others. No missed predictions.

The narrowing assertions are pinned as exact key sets (expect(Object.keys(record).sort()).toEqual([...])), not absence checks — an over-return defect passes any assertion written only as "does not contain X".

Gates run

gate	result
@objectstack/metadata-protocol suite	71 files, 1051 tests, pass
@objectstack/objectql suite	178 files, 3165 tests, pass
@objectstack/rest suite (direct consumer)	82 files, 1341 tests, pass
tsc --noEmit (metadata-protocol)	63 errors, identical count at base SHA — all pre-existing, in unrelated test files; none in protocol.ts
eslint --no-inline-config on both changed files	clean

Consumption radius

assertProjectionFieldsExist is private with exactly two call sites, both in protocol.ts — findData (the list door: POST /query body fields, GET ?$select=/?select=, all folded into fields by WIRE_QUERY_ALIAS_SLOTS before the gate) and getData (GET /:object/:id?select=). Both card doors verified to route through it. No external callers.

Swept packages/ and examples/ for non-test source passing a dotted projection: none. Nothing in-tree breaks.

Not measured

Mongo and the memory driver (both resolve dotted paths against the row, where a FK is a scalar id — likely undefined rather than a widening, but I did not drive them). Live HTTP; I measured at protocol.findData and at driver.find, not over the wire. Showcase seed data.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TXK7QuSmTF3DKWji91cmkn

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
objectstack	Ignored		Aug 11, 2026 6:34am

[github-actions]

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/metadata-protocol.

3 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

• content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/metadata-protocol)
• content/docs/kernel/services-checklist.mdx (via @objectstack/metadata-protocol)
• content/docs/protocol/kernel/http-protocol.mdx (via @objectstack/metadata-protocol)

⛔ 1 release-owned page(s) also reference the affected code. These are read-only:

• content/docs/releases/v9.mdx (via @objectstack/metadata-protocol)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.