[#313] feat: trigger/effectiveness eval harness + conformance (task T7)#324
Conversation
- eval prompt sets (79 prompts, 12 families, all 35 skills) + runner README - baseline run committed: 76 PASS / 3 AMBIGUOUS (all assess-* overlap — motivates T3) - scripts/skills-conformance-check.js: frontmatter portability, name+description <=1024, pointer resolution, catalog counts (warning-only until #313/T1) - wired as skills:conformance in quality-gate + CI; 18 unit tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t (review) - port scripts/skills-conformance-check.js -> packages/knowledge-hub/src/skills-conformance-check.ts (typed ESM module, check-broken-links.ts style; require.main CLI guard prints same PASS/WARN/FAIL) - sibling test imports the module directly (drop createRequire bridge); 18 cases unchanged - package script skills:conformance = ts-node src/skills-conformance-check.ts (mirrors check:links) - root skills:conformance delegates to the package; quality-gate + CI unchanged - delete root script (single source of truth); eval README path ref updated Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restructure: conformance logic moved into the tested module (review follow-up)Addressed the fair objection that the conformance logic lived in a root What changed
Why this dissolves the concern
Gates (all green) The 3 WARNs (next's stated 33-skill catalog vs the real 35) are unchanged and remain warning-only with |
Code Review — PR #324Review InformationPR Number: #324 Review SummaryOverall Assessment
Key Changes SummaryT7 delivers: (a) the trigger-eval prompt set Business Value ValidationSatisfies AC7 (before-evidence + documented eval procedure — the "after" half correctly deferred to T3 per the dependency graph) and AC8's conformance requirement (portability, ≤1024 limits, pointer resolution, catalog counts) wired into CI. Delivers the gate that GATES T3. Verification performed (independent, at head
|
| Check | Result |
|---|---|
pnpm skills:conformance (root → package delegation) |
PASS — 35 skills, exactly 3 WARN (next 33-vs-35), exit 0 |
pnpm --filter @pair/knowledge-hub test |
98/98 pass (incl. skills-conformance-check.test.ts 18/18) + check:links valid |
pnpm --filter @pair/knowledge-hub ts:check |
PASS |
pnpm --filter @pair/knowledge-hub lint |
PASS |
pnpm docs:staleness |
PASS — 35 skills, 8 commands in sync |
| Real-violation exit code (fabricated corpus) | exit 1 for assistant-specific field + over-limit description (+combined bound) + dead pointer |
Restructure correctness (scripts/*.js → tested module + thin CLI)
- Behavioral parity — CONFIRMED. Function-by-function diff of the deleted
scripts/skills-conformance-check.js(e705370) vs the new module:parseFrontmatter,checkFrontmatterFields,checkSizeLimits,extractLinkTargets,isCheckableTarget,checkLinks,checkCatalogCounts,collectSkillFiles,runChecks,SPEC_FIELDS,PAIR_EXTENSIONSare semantically identical. Only intra-refactor: quote-stripping extracted intounquote()(same logic). All four checks preserved (frontmatter portability incl. assistant-specific=error; name≤64 / description≤1024 / combined≤1024; fenced-code + placeholder-skipping pointer resolution; catalog-count WARN). - CLI + path resolution — CONFIRMED. New
ROOT = join(__dirname,'..')→packages/knowledge-hub,SKILLS_DIR = dataset/.skills→ resolves to the samepackages/knowledge-hub/dataset/.skillsas the oldscripts/-relative path.require.mainguard runs correctly underts-node; identical PASS/WARN/FAIL output + exit codes. Verified by running it. - Test rewire — CONFIRMED.
createRequire('../../../scripts/...')bridge +ConformanceModuleinterface shim removed; nowimport { … } from './skills-conformance-check'. Diff shows that is the ONLY change to the test — every case survives (parseFrontmatter, checkFrontmatterFields incl.disable-model-invocationrejection, checkSizeLimits incl. combined bound, isCheckableTarget, extractLinkTargets fenced-code skip, checkCatalogCounts, runChecks fixture corpus). 18/18 green. - Single source of truth — CONFIRMED.
scripts/skills-conformance-check.jsdeleted; repo grep (excl. node_modules/.git) forscripts/skills-conformance-checkandskills-conformance-check.js→ zero dangling refs. Rootskills:conformancedelegates to the package;quality-gatekeeps it in the same position (afterdocs:staleness, beforedup:check); CI adds a step running the unchangedpnpm skills:conformanceright after docs-staleness. The net diff vsorigin/mainreflects the delegation state (what merges), not the intermediate script. - Layout — CONFIRMED. Module + sibling test flat in
src/, mirroringcheck-broken-links.{ts,test.ts}; no package-root test, no subfolder.
Original T7 substance
- Prompt sets — CONFIRMED. 79 prompts (53 should-trigger + 26 should-not-trigger), 12 families,
corpus_size=35=actual SKILL.md count. Every one of the 35 corpus skills has ≥1 should-trigger prompt; no expected/near_miss_for/family skill falls outside the corpus (one intentional family glob — see F1); no duplicate ids. Should-not-trigger prompts are genuine sibling near-misses (setup-pm vs assess-pm, review vs verify-done, estimate vs refine-story, plan-tasks vs write-issue, verify-quality vs assess-code-quality, …). - Baseline — CONFIRMED. Methodology disclosed (single LLM, single pass, catalog-only cold start, author=executor bias disclosure). Stats internally consistent: 51/53, 25/26, 76/79 all ≈96.2%; 3 AMBIGUOUS = AD-12, AD-14, PB-N2, matching the per-prompt tables. The assess-* ambiguities trace accurately to the real current descriptions (assess-pm "proposes a tool choice" vs setup-pm "guiding selection"; assess-stack "implementation (add dependency)" vs verify-adoption "checks code and configuration against adoption files").
- README vs principle 9 — CONFIRMED. README aligns with
writing-skills.mdx§9 Evaluation ondocs/#313-authoring-standard(hypothesis/experiment framing, fresh-session cold start, before/after comparison, and the regression⇒revert-that-description rule). Correct post-restructure module path referenced. - WARN-not-error scoping — CONFIRMED empirically. Catalog-count is warning-only (
TODO(#313/T1), does not affect exit). A fabricated corpus with an assistant-specific field, an over-limit description, and a dead pointer produced 4 errors and exit 1 — the hard rules genuinely fail the gate. Fixture test corroborates.
Detailed Review Comments
Positive Feedback
- The restructure fully dissolves the original cross-boundary-test objection: logic and test are now genuine siblings, byte-for-byte behavior preserved, CLI reduced to a thin caller — verified by running the full gate, not just reading it.
- Prompt set is disciplined: complete skill coverage, realistic phrasing, decoys that name the sibling they target and the correct destination; the baseline is honest about author=executor bias and marks near-ties rather than silently passing them.
- WARN/error split is exactly right: the one known drift (next 33-vs-35) is a non-blocking warning tied to T1, while portability/limits/pointer violations hard-fail — proven empirically.
Minor Issues (non-blocking)
- eval/trigger-prompts.json (AD-N3) —
near_miss_for: "assess-*"uses a family glob rather than a concrete skill name, unlike every other entry. Semantically clear (family-level decoy,expected: none, discussed in the baseline) but schema-inconsistent. nonActionable — intentional family decoy; documented. Recommendation (optional): note the glob convention in the file's$comment. - eval/results/2026-07-12-baseline.md (Ambiguities section) — prose says "Both ambiguities … (AD-12, PB-N2, AD-14)" while listing three ids; it groups two root-cause kinds, but the wording reads as a miscount. The summary stats table is correct (3 AMBIGUOUS). Recommendation (optional): reword to "The ambiguities … fall into two kinds".
- eval methodology (baseline.md) — single-model, single-pass, author==executor. A real self-grading / no-inter-rater-reliability limitation, so the 96.2% is a soft number and the before/after gate's reliability rests on disciplined manual re-runs. nonActionable — explicitly disclosed; AC7 asks for a documented procedure + recorded evidence, both present; appropriate for a "before" baseline.
- src/skills-conformance-check.ts:
extractLinkTargets— strips only triple-backtick fenced blocks, not inline code spans, so a link inside`…`would still be pointer-checked. nonActionable — inherited unchanged from the deleted script (behavioral-parity requirement); no current false positive (conformance PASSes clean).
Questions
- None blocking. (Scope note, not a defect: this PR ships only the "before" baseline; the "after" run lands with T3 per the dependency graph — correct.)
Testing Review
New tests are clear, independent, well-named; the fixture-corpus e2e exercises the real runChecks including the negative cases (assistant-specific field, broken link, catalog warning). Coverage of the module's primitives is complete. All 98 knowledge-hub tests pass.
Risk Assessment
| Risk | Impact | Probability | Mitigation |
|---|---|---|---|
| Manual eval not reproducible mechanically | Low | Med | Documented procedure + committed evidence; conformance (the automatable half) is fully gated |
| Catalog WARN masks a future real count drift | Low | Low | Scoped to next/SKILL.md, TODO(#313/T1) to promote to error |
Adoption Compliance
No new dependencies, libraries, or architectural decisions introduced (harness + gate only). No ADR required. No skill/dataset content changed, so mirror byte-consistency is not exercised by this PR.
Decision: Approved with Comments — no critical or major issues; all AC in T7's scope met; restructure verified behavior-preserving. The four Minor notes are non-blocking (three nonActionable / by-design).
…eline wording - git mv 9 files: CLIs+siblings -> src/tools/, dataset-assertion tests -> src/conformance/ - path-depth fixups (one extra ..): tools/ ROOT/DATASET, conformance/ readFileSync dataset paths - package.json check:links/transfer:dataset/skills:conformance -> src/tools/<file>.ts - drop stale src/index.ts from vitest.config exclude - eval baseline: 'Both ambiguities'->'two kinds' (review finding 2); README path Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Role-based
|
| File → new dir | old → new |
|---|---|
tools/check-broken-links.ts |
ROOT = join(__dirname, '..') → join(__dirname, '..', '..') |
tools/transfer-dataset.ts |
DATASET = join(__dirname, '..', 'dataset') → join(__dirname, '..', '..', 'dataset') (+ usage string) |
tools/skills-conformance-check.ts |
ROOT = join(__dirname, '..') → join(__dirname, '..', '..') |
conformance/agents-md.test.ts |
../dataset/AGENTS.md → ../../dataset/AGENTS.md |
conformance/assess-output-only.test.ts |
../dataset/… → ../../dataset/…; mirror ../../../.claude/skills → ../../../../.claude/skills |
conformance/quality-model.test.ts |
all 4 ../dataset/… → ../../dataset/… |
Verified by live run, not assumption:
pnpm skills:conformance(root → package delegation) reports 35 skills (a brokenSKILLS_DIRwould silently report 0) with exactly 3 WARN (next 33-vs-35) and exit 0.- The 3 dataset tests still read real files (their fixtures load via top-level
readFileSync; a wrong path would ENOENT at collection and fail the suite). Counts unchanged: agents-md 1, assess-output-only 24, quality-model 20 — all passing on real content, not vacuous. - The sibling import
./skills-conformance-checkin the conformance test is unaffected (module + test moved together).
Wiring: package scripts check:links / transfer:dataset / skills:conformance now point at src/tools/<file>.ts; root delegation unchanged; CI step unchanged. Also dropped the stale src/index.ts entry from vitest.config.ts coverage exclude (no such file).
2. Review finding 2 (Minor, actionable)
eval/results/2026-07-12-baseline.md: the prose "Both ambiguities … (AD-12, PB-N2, AD-14)" contradicted its own three-id list. Reworded to "the ambiguities fall into two kinds" — the summary table's count of 3 rows was already correct; only the prose is fixed.
Findings 1, 3, 4 are non-actionable / by-design and acknowledged without code change: (1) single-model/single-pass methodology is deliberately disclosed in-file as the baseline's stated method; (3) the assess-* family-glob near-miss (AD-N3) is an intentional decoy proving the bare word "assess" over-activates the family; (4) extractLinkTargets' fenced-code-only exclusion is inherited unchanged for parity with the reviewed logic.
Gates (all green)
@pair/knowledge-hub test: 98/98 (tools/ + conformance/ suites) + check:links valid
@pair/knowledge-hub ts:check: PASS
@pair/knowledge-hub lint: PASS
@pair/knowledge-hub prettier:check: PASS (moved files normalized)
pnpm skills:conformance (root → package): 35 skills, 3 WARN, exit 0
pnpm docs:staleness: PASS — 35 skills, 8 commands
pnpm mdlint:check: PASS
pre-push quality-gate: full run on push
Re-review of reorg
|
Code Review — PR #324 (canonical
|
| Phase | Outcome |
|---|---|
| 1 — Validation | PR open, mergeable; story #313 loaded; type = Testing. |
2 — Technical review + verify-quality |
All gates re-run at 1593ad2, green (table below). No critical/major. |
| 3 — Adoption compliance | No new deps/libraries/architecture. No ADR required. Conformance correctly scoped to dataset/.skills (portable source), not the assistant-specific .claude/skills mirror. PASS. |
4 — Completeness (verify-done + assess-debt) |
T7-scope DoD met; debt surfaced (output-only, non-blocking). |
| 5 — Decision | Approved with Comments. |
| 6 — Merge | NOT run — human merge gate. |
Verification performed (independent, at head 1593ad2)
| Check | Result |
|---|---|
pnpm --filter @pair/knowledge-hub test |
98/98 pass — 6 suites (tools/ + conformance/), incl. skills-conformance-check.test.ts 18/18; check:links valid |
pnpm --filter @pair/knowledge-hub ts:check |
PASS (strict/NodeNext) |
pnpm --filter @pair/knowledge-hub lint |
PASS |
pnpm --filter @pair/knowledge-hub mdlint:check |
PASS |
pnpm skills:conformance (root → package delegation) |
PASS — 35 skills, exactly 3 WARN (next 33-vs-35), exit 0 |
pnpm docs:staleness |
PASS — 35 skills, 8 commands in sync |
| Negative test (fabricated corpus: assistant field + 1100-char desc + dead pointer) | 4 errors → exit 1 — the hard rules genuinely fail the gate |
Prompt-set validated programmatically against the live corpus: 35 skills / declared corpus_size 35; 12 families; 79 prompts (53 ST / 26 SNT — matches baseline); 0 duplicate ids; every one of the 35 skills has ≥1 should-trigger prompt; every expected/near_miss_for ref resolves to a real corpus skill (or none/glob). Baseline stats internally consistent (76 PASS / 3 AMBIGUOUS / 0 FAIL; the 3 ambiguities = AD-12, AD-14, PB-N2, all real assess-* overlap). Behavioral parity of the restructure (deleted scripts/*.js → module) and the path-depth .. fixups from the reorg were confirmed by the live 35-skill run reporting 35 (a broken SKILLS_DIR would report 0) and by all 6 suites reading real fixtures.
Detailed Review Comments
Positive
- Gate is fail-closed on the rules that matter: assistant-specific frontmatter, over-limit name/description, combined>1024, and dead pointers each produce exit 1 (verified live), while the one known drift (next 33-vs-35) is WARN-only with
TODO(#313/T1)— the right split. - Restructure is genuinely behavior-preserving: logic + test are now siblings, the CLI is a thin caller of exported primitives, single source of truth (
scripts/skills-conformance-check.jsdeleted, no dangling refs). The conformance CLI imports onlyfs/path, so the new CI step adds no build-order dependency on@pair/content-ops. - Prompt set is disciplined and honest: complete skill coverage, realistic phrasing, decoys that name their sibling + the correct destination, and a baseline that discloses the author==executor bias rather than hiding it.
Minor Issues (non-blocking)
src/tools/skills-conformance-check.ts:68-83(parseFrontmatter) — size gate bypassable by a legal YAML construct. [NEW — not raised by prior reviews] A block-scalar description (description: >or|+ indented lines) parses to the value">"/"|"(length ~1); the indented continuation lines are skipped as non-top-level.checkSizeLimitsthen measures ~1 char and passes, so an over-limit description written in block style silently satisfies AC8's ≤1024 enforcement. Not currently triggered — the whole corpus uses single-line quoted descriptions (max 528 chars), verified — so this is latent, not an active failure. Recommendation (optional, low priority): either resolve block scalars to their full value before length-checking, or fail-closed by rejecting block-scalarname/descriptionso the size rule cannot be bypassed.src/tools/skills-conformance-check.ts:110-116+ comment L12-13 — spec-attribution inverted in messages. [NEW — minor] The message calls description≤1024 the "spec max" and the combined≤1024 bound "Pair max", but the agentskills.io/Claude constraint the story cites is the combinedname+description ≤ 1024. Behavior is correct (both are enforced); only the attribution in the comment/error string is backwards. Recommendation (optional): swap the "spec"/"Pair" labels.eval/trigger-prompts.json(AD-N3) —near_miss_for: "assess-*"uses a family glob rather than a concrete sibling, unlike every other entry. nonActionable — intentional family-level decoy (expected: none), documented in the baseline; schema-consistent enough. Optional: note the glob convention in a$comment.src/tools/skills-conformance-check.ts:123-131(extractLinkTargets) — strips fenced blocks only, not inline code spans, so a link inside`…`would still be pointer-checked. nonActionable — inherited unchanged from the deleted script (parity requirement); no active false positive (conformance PASSes clean).eval/results/2026-07-12-baseline.md— methodology is single-model, single-pass, author==executor. Self-grading with no inter-rater reliability, so 96.2% is a soft number and the before/after gate rests on disciplined manual re-runs. nonActionable — explicitly disclosed in-file; AC7 asks for a documented procedure + recorded "before" evidence, both present; appropriate for a baseline. The automatable half (conformance) is fully mechanized and gated.next/SKILL.mdstates a 33-skill catalog; corpus is 35 (checkpoint + grill). A real router bug — but it is T1's fix (AC1, [#313] fix: router/catalog + correctness fixes (task T1) #325), not T7's. T7's remit is detection, and it does so correctly as a non-blocking WARN withTODO(#313/T1)to promote to error post-T1. nonActionable here — by design; fixing it in this PR would be out of task scope.
Trivial: the else { /* allow importing */ } no-op at L245-247 is dead but mirrors the check-broken-links sibling convention — leave for consistency.
Carryover / re-verification of prior findings
Prior "finding 2" (baseline "Both ambiguities" vs three ids) is resolved — reworded to "fall into two kinds (AD-12, PB-N2, AD-14)". Mild residual awkwardness ("both" after a 3-id parenthetical) but the summary table's count of 3 is correct; non-blocking.
Testing Review
New tests clear, independent, well-named; the fixture-corpus e2e exercises the real runChecks including negatives (assistant-specific field, broken link, catalog warning). Primitive coverage complete. All 98 knowledge-hub tests pass. test script is vitest run && check:links (no coverage-threshold enforcement in this command), so the new module is not gated on a coverage number — acceptable for this suite's configuration.
Adoption Compliance
Level: inline (no new dependencies to resolve). No libraries/patterns/tech-stack entries added — only package.json script wiring + a CI step. No new architectural decision ⇒ no ADR (the story's ADR obligation is AC2/T2, delivered separately in #319). Conformance scope (dataset/.skills only, not the .claude/skills mirror) is correct: dataset is the portable source that must be assistant-agnostic; the mirror is intentionally Claude-specific. No mirror byte-consistency concern — this PR changes no skill content.
Tech Debt (assess-debt — output-only, non-blocking)
- Latent size-gate bypass via block-scalar frontmatter (finding 1) — low impact/low effort; fold into a future hardening if block-style descriptions ever appear.
next33-vs-35 catalog drift — already ledgered to T1/[#313] fix: router/catalog + correctness fixes (task T1) #325; the gate tracks it as WARN.- Manual (non-mechanized) trigger eval — inherent to LLM trigger testing; disclosed. No action.
None of these block; none warrant CHANGES-REQUESTED.
Risk Assessment
| Risk | Impact | Probability | Mitigation |
|---|---|---|---|
| Over-limit description slips past gate via block-scalar YAML | Low | Low | Corpus uses single-line quoted descriptions; hardening noted (finding 1) |
| Manual eval not mechanically reproducible | Low | Med | Documented procedure + committed evidence; the automatable half is gated |
| Catalog WARN masks a future real count drift | Low | Low | Scoped to next/SKILL.md; TODO(#313/T1) to promote to error |
Decision: Approved with Comments. No critical or major issues; all T7-scope acceptance criteria (AC7 + AC8-conformance) met; restructure and reorg verified behavior-preserving by re-running the full gate set, not by reading alone. Six Minor findings, all non-blocking (two new/actionable — a latent block-scalar size-gate bypass and an inverted spec label; four nonActionable/by-design). Merge remains a human gate (Phase 6 not run).
🤖 Generated with Claude Code
…tribution (review) - parseFrontmatter folds YAML block scalars (>, |, chomping) to real length so an over-1024 block-style description is measured, not read as len ~1 (was bypassable) - negative tests: >1024 block-scalar description -> violation (drives CLI exit 1) + fold unit tests - checkSizeLimits: comment citing principle 8 (per-field 1024 = spec, combined = Pair); labels verified correct, not flipped Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review remediation — two conformance-gate Minors (commit
|
Re-review of hardening
|
…ness # Conflicts: # package.json
1 participant
Testing: trigger/effectiveness eval harness + skills conformance gate (T7)
Story: #313 (T7 — Trigger/effectiveness eval + conformance)
Type: Testing · Priority: P1 · Risk: Low (no skill content changed; harness + gate only)
Motivation
AC7 requires before/after trigger-test evidence for the T3 description rewrites; AC8 requires static conformance in CI. This PR delivers the eval that GATES T3 (per the story's dependency graph) plus the conformance check, and records the baseline ("before") run against the current descriptions. The eval procedure follows principle 9 of the authoring standard (PR #319); the conformance rules follow principle 8.
Changes Made
packages/knowledge-hub/eval/trigger-prompts.json: 79 prompts (53 should-trigger / 26 should-not-trigger) across 12 skill families; every one of the 35 skills has ≥1 should-trigger prompt; near-misses name the sibling they decoy (near_miss_for) and the correct destination. Grounded in the current frontmatter descriptions.packages/knowledge-hub/eval/README.md: catalog extraction, fresh-session cold-start runs, PASS/AMBIGUOUS/FAIL verdicts, before/after comparison, regression ⇒ revert that skill's description (iterate on the description, not the eval).packages/knowledge-hub/eval/results/2026-07-12-baseline.md: per-prompt expected vs selected + summary. 76 PASS / 3 AMBIGUOUS / 0 FAIL (96.2% strict). All 3 ambiguities are assess-* overlap (assess-pm↔setup-pm ×2, assess-stack↔verify-adoption) — the known near-clone problem showing up as designed; the qualitative section documents that ~60% of each assess-* description is trigger-inert scaffold. Methodology (single-model, single-pass, author=executor bias disclosure) stated in-file.packages/knowledge-hub/src/skills-conformance-check.ts(typed module,check-broken-links.tsstyle; run viats-node, exported primitives +require.mainCLI guard): (a) frontmatter portability — agentskills.io-core top-level fields + toleratedversion/authorPair extension (principle 8), assistant-specific fields are errors; (b)name≤64,description≤1024, combined ≤1024; (c) relative references in skill bodies resolve (fenced code + placeholder paths skipped); (d) catalog counts innext's SKILL.md vs real corpus count.skills:conformance = ts-node src/skills-conformance-check.ts(mirrorscheck:links); rootskills:conformancedelegates to the package (pnpm --filter @pair/knowledge-hub skills:conformance); wired intoquality-gate(pre-push) and CI after docs-staleness.packages/knowledge-hub/src/skills-conformance-check.test.ts: sibling test importing the module directly; 18 unit tests incl. end-to-end fixture corpus.Known warning-only rule (documented, not weakened)
Rule (d) currently finds 3 mismatches:
next/SKILL.mdstates a 33-skill catalog, corpus has 35 (checkpoint + grill missing — exactly T1's bug). Implemented as WARN, non-blocking, withTODO(#313/T1)in the script to promote to error once T1 regenerates the catalog. All other rules PASS on the current corpus.Files Changed
New
knowledge-hub/srclayout:packages/knowledge-hub/eval/{README.md,trigger-prompts.json,results/2026-07-12-baseline.md}packages/knowledge-hub/src/tools/skills-conformance-check.{ts,test.ts}(typed module: exported primitives +require.mainCLI guard; test imports the sibling directly)git mv, history preserved):check-broken-links.{ts,test.ts},transfer-dataset.{ts,test.ts}→src/tools/;agents-md.test.ts,assess-output-only.test.ts,quality-model.test.ts→src/conformance/. Path-depth fixups (one extra..) applied to every moved file's dataset/root paths and verified by a live 35-skill conformance run.package.json(skills:conformancedelegates to the package + wired intoquality-gate);packages/knowledge-hub/package.json(check:links/transfer:dataset/skills:conformance→ts-node src/tools/<file>.ts);.github/workflows/ci.yml(CI step — unchanged command, delegates);packages/knowledge-hub/vitest.config.ts(dropped stalesrc/index.tscoverage exclude);eval/results/2026-07-12-baseline.md(review finding 2 wording).scripts/skills-conformance-check.js(logic moved into the module; single source of truth).AC coverage
Testing / Quality Gates
Risk Assessment
Low — additive harness + gate; no skill/dataset content modified. Only failure surface is the new CI step, verified green on the current corpus.
🤖 Generated with Claude Code