Skip to content

harden scaffold-sync: tier-2 CI coverage gate for core exports #865

Description

@vivek7405

Problem

Scaffold-sync enforcement had only tier 1: the .claude/hooks/require-scaffold-with-src.sh commit floor, which blocks a packages/(core|server|cli)/src commit that stages NO scaffold surface. It proves a feature commit touched a scaffold file, but cannot tell a real runnable demo from a doc bullet. That is exactly how #848 shipped forbidden() / unauthorized() documented (app-tree bullets in the scaffold rule files) but with no gallery demo: the floor was green.

Tests are enforced two ways (a commit gate that says "include a test" AND CI that actually runs it). Scaffold-sync was missing that second, un-skippable tier.

Design / approach

Add tier 2 as a CI test that runs under npm test: reconcile the LIVE @webjsdev/core export surface against a hand-curated manifest, and FAIL when a new export is neither demoed in the gallery nor consciously exempted. A new export then turns CI red until it is classified, the analogue of "a test must exist AND pass." Classification is { demo } (a gallery file that references the symbol) or { exempt } with a reason (internal: plumbing, deferred: agent-facing but not yet demoed). The deferred backlog is tracked separately in #859.

Implementation notes (for the implementing agent)

Implemented in PR #861 (this issue retro-tracks it). Key landmarks:

  • Gate: test/scaffolds/gallery-coverage.test.js. A pure reconcile(liveExports, manifest, demoFileExists, demoFileTeaches) returns error strings; failure modes proven with synthetic inputs beside the real-surface assertion.
  • Manifest: test/scaffolds/gallery-coverage.json (100 exports: 12 demoed, 59 internal, 29 deferred).
  • Discovery: scripts/run-node-tests.js walks test/**, so the file runs in CI with no wiring.
  • Landmine: the demo check is a word-boundary match on the symbol, so a demo that only mentions an API in prose still fails unless it truly imports/uses it. render (client entry) and same-named server helpers (cache, revalidate) are exempt, not demoed.
  • Docs surfaces updated: the webjs-scaffold-sync skill (two-tier enforcement section + change-type row), the require-scaffold-with-src.sh header, and agent-docs/framework-dev.md.
  • Respect the prose-punctuation invariant (release: bump core/server/cli versions, honest engines fields #11) in all new text.

Acceptance criteria

  • A new/renamed @webjsdev/core export fails CI until demoed or exempted
  • Counterfactuals prove the gate fires (new export, stale key, missing demo file, prose-only demo, empty reason)
  • Manifest classifies exactly the live export surface (no missing, no extra)
  • notFound converted from prose to a runnable demo (/features/routing/missing renders 404)
  • Enforcement docs updated (skill, hook header, framework-dev.md)
  • Deferred backlog tracked in scaffold: close the coverage-gate deferred gaps (core + server + convention demos) #859

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions