Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 28 additions & 6 deletions content/docs/guide/ci-cd-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ one has its own section below.
| `control-bytes.yml` | Control Byte Scan | Push / PR to `main`, `develop` — **no path filter**; merge-queue builds; manual | **Yes** |
| `docs-links.yml` | Internal Docs Link Check | Push / PR to `main`, `develop` — **no path filter**; merge-queue builds; manual | **Yes** |
| `skills-paths.yml` | Skill Guide Path Check | Push / PR to `main`, `develop` — **no path filter**; merge-queue builds; manual | **Yes** — when a path stated in a `skills/` guide does not exist |
| `skill-examples.yml` | Skill Example Check | Push / PR to `main`, `develop` — **no path filter**; merge-queue builds; manual | **Yes** — when a MARKED fenced example in a `skills/` guide no longer compiles against the packages' built types, no longer parses as JSON, or carries a marker that opts nothing in |
| `skill-examples.yml` | Skill Example Check | Push / PR to `main`, `develop` — **no path filter**; merge-queue builds; manual | **Yes** — when a MARKED fenced example in a `skills/` or `.claude/skills/` guide no longer compiles against the packages' built types, no longer parses as JSON, uses a bare `any`, or carries a marker that opts nothing in |
| `skill-eval-tokens.yml` | Skill Eval Token Check | Push / PR to `main`, `develop` — **no path filter**; merge-queue builds; manual | **Yes** — when an eval assertion's `must_contain` token is not taught as a whole token anywhere in its own `skills/` bundle |
| `doc-component-types.yml` | Doc Component Type Check | Push / PR to `main`, `develop` — **no path filter**; merge-queue builds; manual | **Yes** — when a `content/docs/**.mdx` snippet teaches a `type` nothing registers |
| `doc-snippet-types.yml` | Doc Snippet Type Check | Push / PR to `main`, `develop` — **no path filter**; merge-queue builds; manual | **Yes** — when a covered documentation snippet no longer compiles against the packages' built types |
Expand Down Expand Up @@ -600,13 +600,13 @@ how it was classified.

**Triggers:** Push and PR to `main`/`develop`, merge-queue builds, plus manual dispatch — with **no
path filter at all**, for the same reason as the section above: the scan surface is markdown under
`skills/`, and both `ci.yml` and `lint.yml` list `'**/*.md'` under the `paths-ignore` of their `push`
trigger. It appears in the checks list as **Skill Example Check**.
`skills/` and `.claude/skills/`, and both `ci.yml` and `lint.yml` list `'**/*.md'` under the
`paths-ignore` of their `push` trigger. It appears in the checks list as **Skill Example Check**.

Runs `scripts/check-skill-examples.mjs`. Where the section above checks the *paths* a guide states in
prose, this one checks the *worked examples* themselves: a marked `ts` / `tsx` / `typescript` fence
must compile `--strict` against the packages' built `dist/*.d.ts`, and a marked `json` / `jsonc`
fence must parse.
must compile `--strict` against the packages' built `dist/*.d.ts` **and must not use a bare `any`**,
and a marked `json` / `jsonc` fence must parse.

**Why it was needed:** at the time it landed, `skills/objectui/` carried 112 TypeScript fences and 56
JSON fences and **not one gate in the repository read inside any of them**
Expand All @@ -624,6 +624,27 @@ marker is an inert HTML comment and leaves the fence info string bare, so the th
the info string still see the block. The convention is ported byte-for-byte from objectstack's
`packages/spec/scripts/check-skill-examples.ts`.

**No bare `any` in a marked block** ([#7463](https://github.com/objectstack-ai/objectui/issues/7463)):
a marker is the author's claim that the block compiles, and every property access on an `any` is
unchecked — so a marked block full of `any` is a green badge over a `tsc` run that proved nothing.
The rule is ported from objectstack's runner with its scope intact: the annotation must **be** `any`
in a position that erases checking (a parameter, a variable / property / return annotation, a type
alias, an `as any` / `satisfies any` / angle-bracket assertion). An `any` **nested** inside a larger
type — `Record<string, any>`, `any[]`, `Promise<any>` — is deliberately allowed; that boundary is
what keeps a red meaning broken. The corpus had four such sites when the assertion landed, and each
is declared verbatim in `KNOWN_BARE_ANY_EXAMPLES` in the script. That list is a **shrink-only**
ratchet, not an allowlist: a row whose red goes away fails as **stale** and must be deleted. Fixing a
row is a judgement about the guide (one of them faithfully restates a platform type that really is
`any`), so the rows are declared debt rather than a mechanical unmark.

**It scans `.claude/skills/` too** ([#7463](https://github.com/objectstack-ai/objectui/issues/7463)):
`SCAN_ROOTS` holds `skills` and `.claude/skills`, the same widening `check-skills-paths.mjs` took in
[#7358](https://github.com/objectstack-ai/objectui/issues/7358). When #7251 moved the two
contributor-only guides out of `skills/`, that gate stopped looking at them and nothing turned red.
Widening a root is **not** arming it: opt-in is the design, so this added 9 candidate fences (18 → 20
guides, 112 → 121 `ts` fences) and **zero** marked ones. Adding a marker under `.claude/skills/` is
the surface owner's step.

**Adjacency is strict:** a marker that is not directly above a checkable fence — separated by a blank
line, above a `bash` fence, or left behind when its example was deleted — is an **orphan** and fails
the run. A lenient rule would opt in nothing while its author believed otherwise, which is this
Expand All @@ -644,7 +665,8 @@ the failure shape this gate family exists to prevent
([#4846](https://github.com/objectstack-ai/objectui/issues/4846)).

**If it fails:** it prints `file:line` for every failing fence with the compiler's own diagnostic, or
the JSON parser's message, or the orphan marker's line. Fix the example — or, if it was never meant
the JSON parser's message, or the orphan marker's line, or — for a bare `any` — the position and the
verbatim baseline row to declare if the fix belongs to a later card. Fix the example — or, if it was never meant
to stand alone, remove its marker rather than weakening the gate. Run it locally with
`pnpm check:skill-examples`, `node scripts/check-skill-examples.mjs --list` to see every candidate
fence and its verdict, and `--measure` to judge every candidate whether marked or not.
Expand Down
93 changes: 87 additions & 6 deletions scripts/__tests__/check-skill-examples.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,14 @@ import { fileURLToPath } from 'node:url';
import {
EXIT_CODES,
JSON_FENCE_LANGUAGES,
KNOWN_BARE_ANY_EXAMPLES,
MARKER,
SCAN_ROOTS,
TS_FENCE_LANGUAGES,
bareAnyRowKey,
buildFilterArgs,
fenceSpans,
findBareAny,
listGuides,
parseJsonFence,
scanSkillFences,
Expand Down Expand Up @@ -232,11 +235,14 @@ describe('JSON fences — `json` is strict, `jsonc` is exactly two things looser
});

describe('the scan surface is a decision, stated here rather than read off the walker', () => {
it('walks `skills` and, today, nothing else', () => {
// `.claude/skills/**` is deliberately outside this gate for now, the way
// `check-skills-paths.mjs`'s prefix allow-list was: widening it is a
// separate, visible edit with its own measurement, not a silent drift.
expect(SCAN_ROOTS).toEqual(['skills']);
it('walks the published bundle AND `.claude/skills`, and nothing else', () => {
// objectui#7463 item 3 widened this to `.claude/skills`, the same widening
// `check-skills-paths.mjs` took in objectui#7358 and for the same reason:
// when objectui#7251 moved the contributor-only guides out of `skills/`,
// a gate rooted only at `skills` silently stopped looking at them. It
// stays a stated decision with a measurement, never a silent drift — the
// widening added 9 candidate fences and ZERO marked ones.
expect(SCAN_ROOTS).toEqual(['skills', '.claude/skills']);
});

it('collects every `.md` under the roots, recursively and in a stable order', () => {
Expand Down Expand Up @@ -271,7 +277,11 @@ describe('the scan surface is a decision, stated here rather than read off the w
// nothing at all.
const guides = listGuides(repoRoot) as string[];
expect(guides.length).toBeGreaterThan(5);
expect(guides.every((g) => g.startsWith('skills/'))).toBe(true);
expect(guides.every((g) => SCAN_ROOTS.some((r) => g.startsWith(`${r}/`)))).toBe(true);
// Both roots must actually be non-empty in this checkout, or the widening
// would be a root list nothing reads — the vacuity this leg exists to deny.
expect(guides.some((g) => g.startsWith('skills/'))).toBe(true);
expect(guides.some((g) => g.startsWith('.claude/skills/'))).toBe(true);
});
});

Expand Down Expand Up @@ -474,3 +484,74 @@ describe('the harness is imported, not re-rolled', () => {
expect(source).not.toContain('ts.createCompilerHost');
});
});

/**
* objectui#7463 item 1 — the bare-`any` assertion, ported from objectstack's
* `packages/spec/scripts/check-skill-examples.ts`.
*
* The NEGATIVE half carries the weight. A bare `any` erases checking wholesale,
* but an `any` nested in a larger type is a much broader question with a much
* larger baseline, and a guard that flagged it would red on prose that is not
* wrong — which is how gates get deleted. That boundary is the whole design, so
* it is pinned rather than left to the implementation.
*/
describe('the bare-`any` assertion', () => {
it.each([
['a parameter', 'export function f(ctx: any) { return ctx; }', 'parameter `ctx`'],
['a variable', 'export const x: any = 1;', 'variable `x`'],
['an interface property', 'export interface I { p: any }', 'property `p`'],
['a class property', 'export class C { p: any = 1; }', 'property `p`'],
['a type alias', 'export type A = any;', 'type alias `A`'],
['a return type', 'export function g(): any { return 1; }', 'return type'],
['an arrow return type', 'export const h = (): any => 1;', 'return type'],
['a method signature return', 'export interface J { m(): any }', 'return type'],
['an `as any` cast', 'export const y = ({} as any);', '`as any` assertion'],
['a `satisfies any`', 'export const z = ({} satisfies any);', '`satisfies any` assertion'],
])('flags %s', (_label, code, want) => {
const hits = findBareAny(code) as { where: string }[];
expect(hits.map((h) => h.where)).toEqual([want]);
});

it.each([
['Record<string, any>', 'export const a: Record<string, any> = {};'],
['any[]', 'export const b: any[] = [];'],
['Array<any>', 'export const c: Array<any> = [];'],
['Promise<any>', 'export async function d(): Promise<any> { return 1; }'],
['a union arm', 'export const e: string | any[] = [];'],
['the word "any" in a string or a comment', 'export const f = "any"; // any of them'],
])('does NOT flag a nested `any` in %s', (_label, code) => {
expect(findBareAny(code)).toEqual([]);
});

it('parses as TSX, so a JSX example is not mis-read as a type assertion', () => {
// `compileSnippets` parses every block as TSX regardless of the fence
// label. A guard walking a different tree would be reporting about a
// program `tsc` never judged.
expect(findBareAny('export const El = () => <div className="x">hi</div>;')).toEqual([]);
});

it('yields nothing rather than throwing on a block too broken to parse', () => {
// The `tsc` syntax leg owns that verdict; this guard must not double-report
// it, and must not crash the run either.
expect(() => findBareAny('export const three: = ;')).not.toThrow();
});

it('builds a baseline row key naming the guide, the fence line and the position', () => {
const block = { doc: 'skills/objectui/guides/x.md', fenceLine: 42 };
const finding = (findBareAny('export function f(ctx: any) {}') as { where: string }[])[0];
expect(bareAnyRowKey(block, finding)).toBe('skills/objectui/guides/x.md:42 parameter `ctx`');
});

it('declares its baseline as a shrink-only Set of verbatim rows', () => {
// Every row must be shaped like a key this gate can actually produce, or it
// would sit in the list forever covering nothing — a parked exemption
// wearing a ratchet's clothes.
expect(KNOWN_BARE_ANY_EXAMPLES).toBeInstanceOf(Set);
for (const row of KNOWN_BARE_ANY_EXAMPLES as Set<string>) {
expect(row, `baseline row is not \`GUIDE:LINE POSITION\`: ${row}`).toMatch(
/^[\w./-]+\.md:\d+ .+$/,
);
expect(SCAN_ROOTS.some((r: string) => row.startsWith(`${r}/`))).toBe(true);
}
});
});
Loading
Loading