feat(checks): add model-unknown warn rule for the model frontmatter field#28
Conversation
…ield Validates the optional model: field against a set of known Claude model IDs. Issues a warn (not error) so CI does not break when new models ship. Also documents empty-body and model-unknown in the README rules table.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 66e4d9d01f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| // Known Claude model IDs. Severity is warn (not error) because the list | ||
| // evolves with each Anthropic release; update this set when new models ship. | ||
| const KNOWN_CLAUDE_MODELS: ReadonlySet<string> = new Set([ |
There was a problem hiding this comment.
Accept documented model aliases and inherit
This whitelist makes valid skill frontmatter produce model-unknown warnings: I checked the Claude Code skills frontmatter docs, which say model accepts the same values as /model, or inherit, and the CLI reference documents /model aliases such as sonnet and opus. A skill using model: inherit or model: sonnet will now be reported as a typo, and in --strict CI that false positive becomes a failing run, so the accepted set should include the documented non-ID values (or delegate validation to Claude Code's model parser).
Useful? React with 👍 / 👎.
Why
Skills can specify a
model:field in their frontmatter to pin the Claudemodel used at runtime. A typo there (
claude-sonet-4-6instead ofclaude-sonnet-4-6) causes the skill to silently run on the wrong model orfall back to a default, with no feedback from the model output. This is
exactly the class of silent bug skillcheck is designed to catch.
This PR adds a
model-unknownwarn rule (not error, because the model listevolves with each Anthropic release) that validates the
model:fieldagainst a curated set of known Claude model IDs.
Also fixes a docs omission: the existing
empty-bodyrule was not listedin the README "What it checks" table.
What
KNOWN_CLAUDE_MODELSinsrc/checks.tscovering Claude 3.xand 4.x series model IDs
checkModelUnknowncalled in the per-skill loop insiderunChecksmessage content, and a 3.x series model
empty-bodyandmodel-unknownrowsTests
test/checks.test.ts: 5 new tests formodel-unknownnpm run lint && npm run build && npm testall green locallySelf-merge gate
Generated by Claude Code