Skip to content

feat(lint): check:doc-formula-expressions reads spec TSDoc @example strings (#6763) - #7180

Merged
os-help merged 1 commit into
mainfrom
claude/issue-6763-spec-tsdoc-example-gate
Aug 10, 2026
Merged

feat(lint): check:doc-formula-expressions reads spec TSDoc @example strings (#6763)#7180
os-help merged 1 commit into
mainfrom
claude/issue-6763-spec-tsdoc-example-gate

Conversation

@os-help

@os-help os-help commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Fixes #6763

Landed under the maintainer ruling of 2026-08-09 on that thread — approved as a scan-surface extension, not new machinery, with two conditions ruled in. Maintainer directive verbatim: 「同意你的建议」.

Premise re-verified on origin/main @ 5087ac6

Holds in full. The baseline gate is green at 22 record-scoped formula example(s) across 387 files / 1397 TS blocks, and it never opens a packages/spec/src file. 424 TSDoc @example tags live under packages/spec/src/** and nothing reads any of them.

Three measurements, two of which contradict the obvious design

The PM's dispatch flagged one hypothesis as most worth falsifying — that the judging half is reusable but the extraction half is not. Confirmed, and then some.

1. Extraction is genuinely new. #6641's defect was not in a fenced block at all. It was an inline @example "EXPR" - caption string hanging off a property:

   * @example "status IN ('draft', 'pending')" - Only allow certain statuses

Adding packages/spec to ROOTS would have read exactly nothing: the walker collects .md/.mdx only, and the extractor reads markdown fences.

2. The existing admission rules admit ZERO on this surface. All 424 @example tags were run through the current extractFormulaExpressions, both fenced-only and whole-body: 0 admitted, 0 tripwires. There is no Field.formula(…) or type: 'formula' example in the spec sources today.

This is the load-bearing measurement. The ruling's literal instruction — reuse the existing admission logic — produces a gate that admits nothing, i.e. a no-op that arrives green and meaningless, which is the one thing condition 2 forbids ("so the gate arrives green and meaningful"). Pass A below still applies that rule to the new surface, because it is the ruling's letter and it costs nothing; pass B is what makes the gate mean something.

3. Scope cannot be inferred from the slot's schema type. 25 slots are typed ExpressionInputSchema, and they do not share a scope: hook.condition binds the record, page.visibleWhen also binds page variables, flow.condition is flow-scoped. Judging them all as record-scoped reports this measured verdict on a correct page predicate:

source validateExpression(…, { scope: 'record' })
page.selectedProjectId != '' bare reference `page` … Write `record.page`.

That is precisely the false-red the script's existing discriminator note exists to prevent, one surface over.

What changed — one file, packages/lint/scripts/check-doc-formula-expressions.mjs

Pass A — the ruling's letter. The existing record-scoped formula discriminator, pointed at fenced TS blocks inside spec TSDoc @example bodies. Same admission, same verdict, new surface. Admits 0 today; kept because the rule is live on surface 1 and the day a spec docblock grows a Field.formula({ expression: 'qty * price' }) example it is judged rather than shipped.

Pass B — a declared slot registry, SPEC_EXAMPLE_SLOTS: a (declaration, property) pair, its dialect, and the imported verdict for that dialect.

entry slot dialect verdict, imported from @objectstack/formula
rls-predicate RowLevelSecurityPolicySchema.using / .check RLS predicate (ADR-0056 D4) isSupportedRlsExpression
hook-record-condition HookSchema.condition record-scoped CEL (ADR-0058 D1) validateExpression(…, { scope: 'record' })

No verdict is reimplemented — the RLS red message computes its reason via isPushdownableCel(sqlPredicateToCel(…)) only after isSupportedRlsExpression has already said no, so it is decoration on a decided verdict rather than a second opinion (Prime Directive #12).

The registry cannot rot quietly in either direction:

  • an entry matching no site is an error — rename the schema and the gate says so instead of shrinking in silence, the same reasoning as assertRootsResolvable;
  • a slot typed by an expression-input schema that carries an @example but is not registered is an error naming the slot. That is the tripwire for the next expression slot someone documents: its scope is not knowable from its type, so the gate refuses to guess and asks for a registry entry instead.

Pass A is a text walk and pass B is an AST walk, and they disagree on purpose: ts.getJSDocTags only reaches comments the parser attached to a node, which is 110 of the 424 tags — the fenced typescript examples live on module docblocks and on the schema declaration itself, which pass A must see. Pass B needs the node to know the slot, so it is AST-bound by nature.

Condition 1 — the day-one exemption idiom

EXEMPT_EXAMPLES, same discipline as the parity-gate lists and check-error-code-casing.mjs's EXEMPT_FILES: an entry carries the site and its reason, never a blanket path ignore. Keyed on the example's exact text rather than a line number, so it survives reflow but dies with the example it excuses.

Self-invalidating in both directions, which is what separates "we thought about this one" from a hole:

  • an entry matching no site is an error (stale);
  • an entry whose site now judges clean is an error (unnecessary — delete it and let the rule cover the site again).

All three directions plus an exact-match near-miss are executed in the self-test, not described.

Condition 2 — the stock pass, and its count

✓ check:doc-formula-expressions: 22 record-scoped formula example(s) across 387 files / 1402 TS blocks judged clean by @objectstack/formula.
✓ check:doc-formula-expressions (spec TSDoc, #6763): 8 @example(s) judged clean across 698 packages/spec/src files — rls-predicate=8, hook-record-condition=1, record-formula=0; 1 exempt.

9 sites admitted on the new surface, of which 1 is red:

rule sites verdict
rls-predicate 8 all clean (5 using, 3 check — post-#6729)
hook-record-condition 1 RED — exempted, see below
record-formula (pass A) 0 an honest zero, see the positive control

The gate arrives green.

The red one: a second live instance of the class

packages/spec/src/data/hook.zod.ts:236 documents a record-scoped CEL slot with a SQL predicate:

   * @example "status = 'active' AND amount > 1000"
   */
  condition: ExpressionInputSchema.optional().describe('Predicate (CEL); hook runs only when TRUE. e.g. P`record.status == "closed" && record.amount > 1000`'),

The @example and the .describe() are one line apart and disagree about the dialect. Measured: invalid CEL value: Unexpected character: =. It is wrong twice over — even after the operators are fixed, bare status / amount would still resolve to nothing and evaluate silently to null (the #5116 shape).

This is a second live instance of the class, found by measurement rather than by hand, which is what the findings-round grading on #6763 named as this class's promotion condition.

Not fixed here, deliberately. packages/spec/src/** belongs to the domain:spec / domain:spec-surface seats and this card is domain:spec-tooling, so per the dispatching seat's boundary constraint it is exempted with a reason naming it as a real defect pending cross-seat transfer, and filed as #7175. The ruling's "arrives green" condition holds; the difference is that a real defect leaves a labelled trail instead of being silently repaired by the wrong seat. Deleting that exemption is part of #7175's fix, and the "unnecessary" direction above means it cannot be forgotten silently.

The positive control — why record-formula=0 is not a blind scanner

A scanner that silently sees nothing reads identically to a clean corpus, so the zero is proven rather than asserted. #6641's string was re-injected into an in-memory copy of the real rls.zod.ts (the file on disk is never touched) and the production extractor re-run over it:

BEFORE #6729 (the defect as it shipped)
  extracted 8 RLS @example(s) from the REAL file; 1 would be REJECTED
    x packages/spec/src/security/rls.zod.ts:383 [RowLevelSecurityPolicySchema.check] "status IN ('draft', 'pending')"  -> bridged "status in ('draft', 'pending')"

AFTER #6729 (main today)
  extracted 8 RLS @example(s) from the REAL file; 0 would be REJECTED

The extractor reaches the real file, reads all 8 examples out of it, and catches the historical defect. The zero on record-formula is therefore a fact about the corpus, not about the scanner. The substitution itself carries an anti-vacuity throw: if the #6729 fix line is not found verbatim in the real source, the control fails rather than reporting a happy zero.

Reverse verification — direction predicted before running, both RED

1. Delete the exemption — the hook defect surfaces, proving the exemption is load-bearing and the red is real:

✗ check:doc-formula-expressions — 1 spec TSDoc @example(s) would be
  REJECTED by the runtime that compiles them:

    packages/spec/src/data/hook.zod.ts:236  [HookSchema.condition — record-scoped CEL (ADR-0058 D1)]
      source: "status = 'active' AND amount > 1000"
      invalid CEL value: Unexpected character: =
      >    1 | status = 'active' AND amount > 1000
                      ^ — values are bare CEL (e.g. `record.rating >= 4`).

2. Rename a registry entry's declaration — the anti-vacuity guard fires instead of the scan silently shrinking:

✗ check:doc-formula-expressions — 1 SPEC_EXAMPLE_SLOTS entr(y/ies) matched
  NO @example anywhere under packages/spec/src: rls-predicate

Both were taken out with a file copy, never git stash.

Self-test — 11 cases to 24

The 11 existing cases are untouched and still pass. 13 added, the RED ones verbatim from the real defects so this fails if the gate ever stops catching them:

  ✓ RED — #6641 verbatim: the RLS `check` example that compiles to a deny-everything policy
  ✓ GREEN — the shipped fix (#6729): the same idiom as a CEL bracket list
  ✓ GREEN — the `using` examples the SQL bridge still carries, caption and all
  ✓ RED — the record-scoped hook `condition` example is SQL where the slot is CEL (#6763 stock pass)
  ✓ NOT ADMITTED — a neighbouring slot's JSON @example is not an expression
  ✓ LOUD — an expression-typed slot nobody registered is reported, never guessed at
  ✓ LOUD — a registered slot whose @example is prose carries no judged expression
  ✓ PASS A — a fenced formula example inside spec TSDoc is judged by the existing rule
  ✓ PASS A — the canonical spelling in a fenced spec example is clean
  ✓ EXEMPTION — a red site with a matching entry is cleared, and nothing is stale
  ✓ EXEMPTION — an entry that matches no site is STALE, not a silent pass
  ✓ EXEMPTION — an entry over a site that now judges clean is UNNECESSARY
  ✓ EXEMPTION — the match is exact: a near-miss source does NOT excuse the site

✓ check:doc-formula-expressions self-test: 24 cases passed

CI cost

Measured on the same container, same corpus: 1.23s before, 2.71s after — about +1.5s for 698 additional files parsed. Reported rather than worked around; the scan root was not narrowed to make it look cheaper.

Verification

gate result
pnpm --filter @objectstack/lint run check:doc-formula-expressions self-test 24/24 + both scans green (output above)
pnpm --filter @objectstack/lint test Test Files 68 passed (68) / Tests 1771 passed | 4 skipped (1775)
pnpm --filter @objectstack/lint typecheck clean
npx eslint packages/lint/scripts/check-doc-formula-expressions.mjs clean
node scripts/check-nul-bytes.mjs OK (scanned 6593 text file(s) … no raw ASCII control bytes)
self-scan grep -naP over the control-byte class on the changed file no hits
pnpm --filter @objectstack/spec check:generated --reconcile-only 20 check: + 14 gen: scripts, all classified (no script entry added or renamed)

Scope notes

  • One file changed. packages/spec/src/** is byte-identical — confirmed by git status. No .zod.ts example was edited in this PR.
  • No changeset, hence skip-changeset: packages/lint's published files are ["dist", "README.md", "CHANGELOG.md"], so scripts/ ships in no package. This is a CI gate only and releases nothing.
  • The script keeps its name. It now covers a second dialect on a second surface, which makes doc-formula-expressions a little narrow, but renaming the entry would touch lint.yml and the check:generated ledger — wider than "a scan-surface extension" and a separate call. The header says so plainly rather than leaving the name to mislead.
  • content/docs/releases/ untouched.

Generated by Claude Code

… strings (#6763)

A TSDoc `@example` is not reachable from any import, so no ordinary test can go
red on it and — until now — no gate read one. #6641 was the proof: the RLS
`check` slot documented its enumerated-values idiom as
`"status IN ('draft', 'pending')"`, which does not compile, so an author copying
the schema's own example got a policy that denies every row. It was found by
hand, months late.

Landed as the ruled scan-surface extension: the judgment stays imported from
`@objectstack/formula`, only the surface is new. Three measurements shaped it,
two of them against the obvious design:

- the extraction half is genuinely new — #6641's defect was an INLINE
  `@example "<expr>" - caption` on a property, not a fenced block, so pointing
  ROOTS at packages/spec would have read nothing;
- the existing admission rules admit ZERO across all 424 spec `@example` tags,
  so a gate carrying only them would arrive green and meaningless;
- scope is not inferable from the slot's schema type — 25 slots are typed
  ExpressionInputSchema and they do not share a scope, so judging them all as
  record-scoped would false-red a correct page predicate.

So admission is a declared slot registry (2 entries, both measured), which
cannot rot quietly: an entry matching no site is an error, and an
expression-typed slot carrying an @example that no entry claims is an error
naming the slot rather than a guess at its scope.

Exemptions are named, reasoned and self-invalidating in both directions —
stale (matches no site) and unnecessary (the site now judges clean) are each
an error.

Stock pass, per the ruling's condition 2: 9 sites admitted — rls-predicate=8
(all clean), hook-record-condition=1 (RED, exempted and filed as #7175 because
packages/spec/src is another seat's territory), record-formula=0. The zero is
proven honest by a positive control that re-injects #6641's string into an
in-memory copy of the real rls.zod.ts and watches the extractor catch it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJATVrh6V2ysutYUJigh3B
@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 10, 2026 1:57am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

No hand-written docs reference the 0 changed package(s). ✅

@os-help os-help added skip-changeset PR has no user-facing published change; bypasses the changeset gate and removed size/l labels Aug 10, 2026 — with Claude
@os-help
os-help marked this pull request as ready for review August 10, 2026 02:23
@os-help
os-help added this pull request to the merge queue Aug 10, 2026
Merged via the queue into main with commit 775750e Aug 10, 2026
35 of 36 checks passed
@os-help
os-help deleted the claude/issue-6763-spec-tsdoc-example-gate branch August 10, 2026 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No gate reads spec TSDoc @example strings, so an expression example that stops compiling is invisible — #6641 was found by hand

2 participants