Repository navigation
Patch release. Commands and flags are unchanged.
Fixed
Review gate
- A recorded independent-review skip (
activation.required: falsewith a reason) no longer failsscripts.validateorscripts.review_gatewithreview-agent-missingon a skill that does not ship the toolkit'sagents/*.mdfiles. The skip is still validated for a reason, a source binding, and staleness. A skip without a reason does not waive the agent check, and a required review still needs the agent files. Reported by @chaser940428-pixel.
Evaluation engine (five correctness fixes)
-
Behavior grading scoped to one case —
grade_behavior()now accepts--behavior-caseand passes only the selected case's assertions to the judge. Multi-case suites require explicit selection; single-case suites auto-select. Duplicate assertion text across different cases is now permitted. -
Grades bound to their evidence —
evidence_digest()computes a SHA-256 over assertions, output files, and optional transcript.merge_grades()rejects stale model grades whose digest does not match current evidence. Legacy grades without a digest are flaggedunverified: truerather than silently accepted. -
Blank prompts rejected before dispatch —
run_eval()raisesValueErrorfor blank/whitespace-only prompts before any infra or model call.generate_tests.pyalso filters blank prompts before writing generated YAML fixtures. -
Frontmatter identity used consistently —
grade_behavior()provenance andbsc evalnow use thename:field fromSKILL.mdinstead of the directory name. Baselines tied to a directory name need theirskill_namefield updated to the declared frontmatter name. -
Infrastructure failure diagnostics preserved —
_check_infra()returns a fullQueryOutcomeinstead of a barebool. Failure reports includeinfrastructure_error(e.g.timeout,authentication,not_triggered) andinfrastructure_error_detail.run_looppropagates both fields;held_out_scoreisnullwhen the held-out eval itself hits an infra failure.
Documentation
docs/MODEL_BEHAVIOR.mdnow covers all four current model tiers (Opus 5.5, Sonnet 5.5, Sonnet 5, Haiku 5.5, Fable 5.2) with a tier overview, per-behavior tables linked to Anthropic's own pages, and one-run live trigger snapshots per alias.- Adds a Sonnet 5.5 table built from Anthropic's own prompting guide (effort,
between_toolsthinking, early stopping and verification, mid-turn messages, JSON tasks, tool-call handling). - Corrects an earlier claim that Anthropic had published no Sonnet 5.5 guide. Records what two independent evaluators (Artificial Analysis and Vals.ai) report about Sonnet 5.5, with the conditions each states.
License
The project is now licensed under the Apache License 2.0 with an Additional Terms section (permitted under Apache 2.0 §4(d)): anyone who sells or commercially distributes the Work or any Derivative Work must credit RollingCodes as the original creator with a link to this repository. Free use, modification, and redistribution are unaffected.
Thank you, @chaser940428-pixel
I hope you will contribute to this project again. It took me a long time; I hope you get use out of it. It might look like generic instructions, but it's not. I actually did hours and hours of research for it, but I really, really appreciate your contribution. I don't have as much time as I would like to work on it anymore, so I welcome and appreciate any contributions.
Full changelog: https://github.com/rolling-codes/better-skill-creator/blob/main/CHANGELOG.md#375---2026-11-10