feat(tools): add deterministic skill validator for CI

[alexeyv]

Summary

• Add tools/validate-skills.js — Node CLI that checks 13 deterministic rules (SKILL-01–06, WF-01–02, PATH-02, STEP-01/06/07, SEQ-02) across all skill directories in under 1 second
• Add validate:skills npm script to quality chain and push gate
• Update tools/skill-validator.md with first-pass integration instructions so the inference validator skips already-verified rules
• Includes review hardening: empty value guards, nested YAML protection, safe file reading with error recovery

Test plan

• node tools/validate-skills.js --strict exits 0 (3 true-positive MEDIUM/LOW findings)
• node tools/validate-skills.js --json produces valid JSON
• npm run quality includes and passes validate:skills
• Full test suite (232 tests), ESLint, Prettier, markdownlint all pass
• Verify CI quality workflow passes

🤖 Generated with Claude Code

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR introduces a deterministic skill validator script (tools/validate-skills.js) as a first-pass validation layer, integrates it into the build pipeline via npm scripts, updates documentation to describe a two-pass validation workflow combining deterministic and inference-based checks, and updates references to the validation process.

Changes

Cohort / File(s)	Summary
Documentation AGENTS.md, tools/skill-validator.md	Updated to reflect new deterministic validation workflow. AGENTS.md replaced outdated skill-validator reference with npm script info. skill-validator.md expanded with two-pass validation guidance (deterministic first-pass + inference second-pass), 13 enumerated rules, rule-specific detection/fix details, and restructured formatting for reports and cheatsheet.
Build Configuration package.json	Added validate:skills npm script and integrated it into the quality script to run deterministic checks during builds.
Deterministic Validator tools/validate-skills.js	New 705-line validator implementing 13 deterministic validation rules (SKILL-01 through STEP-07, SEQ-02, WF-01/WF-02, PATH-02). Includes skill discovery, frontmatter parsing (single-line and multiline), rule-based validation, and dual output formats (human-readable and JSON with GitHub Actions integration). Exports four public functions: parseFrontmatter, parseFrontmatterMultiline, validateSkill, and discoverSkillDirs.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

• PR #1996: Applies fixes for installed_path and intra-skill path variables that the new deterministic validator rules explicitly forbid.
• PR #1991: Removes installed_path and intra-skill variables to satisfy PATH-02/PATH-04 checks implemented in the new validator.
• PR #1981: Updates skill validation tooling documentation; this PR integrates the new deterministic validator into the two-pass validation workflow.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat-deterministic-skill-validator

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

tools/validate-skills.js (1)

211-229: Consider adding error handling for directory read failures.

Unlike discoverSkillDirs (which checks fs.existsSync before reading), collectSkillFiles doesn't guard against readdirSync failures. If a directory becomes inaccessible mid-walk (e.g., permission change, symlink target removed), this would throw an unhandled exception.

🛡️ Optional: wrap readdirSync in try-catch

 function collectSkillFiles(skillDir) {
   const files = [];

   function walk(dir) {
+    let entries;
+    try {
+      entries = fs.readdirSync(dir, { withFileTypes: true });
+    } catch {
+      return; // Skip inaccessible directories
+    }
-    const entries = fs.readdirSync(dir, { withFileTypes: true });
     for (const entry of entries) {
       if (entry.name === 'node_modules' || entry.name === '.git') continue;

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tools/validate-skills.js` around lines 211 - 229, The collectSkillFiles
function currently calls fs.readdirSync inside the nested walk function without
handling errors, so a mid-walk permission or IO failure will throw; wrap the
fs.readdirSync(dir, { withFileTypes: true }) call in a try-catch inside walk (or
check fs.existsSync/permissions before reading) to catch and log or silently
skip unreadable directories, and ensure the walk continues for other entries;
update collectSkillFiles/walk to handle and optionally report errors (e.g.,
using console.warn or the existing logger) while still returning the collected
files.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tools/validate-skills.js`:
- Around line 211-229: The collectSkillFiles function currently calls
fs.readdirSync inside the nested walk function without handling errors, so a
mid-walk permission or IO failure will throw; wrap the fs.readdirSync(dir, {
withFileTypes: true }) call in a try-catch inside walk (or check
fs.existsSync/permissions before reading) to catch and log or silently skip
unreadable directories, and ensure the walk continues for other entries; update
collectSkillFiles/walk to handle and optionally report errors (e.g., using
console.warn or the existing logger) while still returning the collected files.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: adc86a9f-0e0a-4565-99ff-8f68de7fdbd3

📥 Commits

Reviewing files that changed from the base of the PR and between 0380656 and 5a1f356.

📒 Files selected for processing (4)

• AGENTS.md
• package.json
• tools/skill-validator.md
• tools/validate-skills.js

[augmentcode]

🤖 Augment PR Summary

Summary: This PR introduces a fast, deterministic skill validator to complement the existing inference-based validation and make CI enforcement more consistent.

Changes:

• Added tools/validate-skills.js, a Node CLI that validates 13 deterministic rules (SKILL-01–06, WF-01–02, PATH-02, STEP-01/06/07, SEQ-02) across all discovered skills.
• Implemented both human-readable and --json output modes, plus --strict behavior that fails the process only on CRITICAL/HIGH findings.
• Integrated the new validator into the quality chain via npm run validate:skills and appended it to npm run quality.
• Updated tools/skill-validator.md to document the deterministic “first pass” and how to skip already-verified rules during inference-based review.
• Documented the new deterministic validation entrypoint in AGENTS.md.

Technical Notes: The validator discovers skill directories under src/core/skills, src/core/tasks, src/bmm/workflows, and src/bmm/agents, and emits GitHub Actions annotations/step summaries when running in CI.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.