Skip to content

v0.8.0

Latest

Choose a tag to compare

@github-actions github-actions released this 22 Sep 17:13

Fixed

  • ai-forge-judge: the description colon rule was wrong. It auto-failed S1 on any colon, but a colon inside a quoted scalar (description: "… animate:flip …") is valid YAML and the agentskills spec imposes no colon rule of its own. Only a bare key: value shape in an unquoted value breaks parsing, and only that is now an auto-fail. The old rule mis-graded correct skills.
  • ai-forge-judge, ai-forge-review: evals/ and benchmarks/ were flagged as skill-root spec violations. A purpose-named directory is not a stray file; only non-SKILL.md files at the root are. Both now say so explicitly, so an eval suite no longer costs a skill points.
  • ai-forge-update: Phase 3 invoked ai-forge-apply unconditionally while the judge's report already ended in its own (y)es / (n)o prompt, so the user was asked twice. Phase 3 now honours that prompt and defines the (n) branch.
  • ai-forge-eval: an agent's result and its <usage> block can arrive in two separate messages. A hand-back without usage was being treated as permanently missing data; it is now recorded as null and overwritten if a later notification for the same task id carries the block.

Changed

  • ai-forge-apply: one decision per turn. The uncommitted-changes warning, the git opt-in, the branch question, and (A)/(s) are four turns rather than one line — a human facing two prompts answers both in one reply ("n A"), which the skill had no defined parse for. Added as a NEVER with that rationale.
  • ai-forge-apply: the git branch decision moved to references/git.md, loaded only on (c)ommit. It also gains an escape hatch — a repo whose own convention puts AI work on the default branch commits there instead of forcing an ai/* branch.
  • ai-forge-apply: (r)evise now collects one sentence about what was wrong with the change, not with the finding.
  • ai-forge-judge: dimension U3 renamed Anti-Pattern Quality → Constraint Quality; the report footer is now the literal prompt Step through these with ai-forge-apply? (y)es / (n)o.
  • ai-forge-eval: trigger validation runs each query twice, with a third run only where the two disagree. A measured 12-query set returned unanimous judgments on all 24 runs, so a blanket third run mostly re-buys an answer already in hand.
  • ai-forge-create: benchmarks/ added to the directory taxonomy for committed trend baselines.

Added

  • ai-forge-apply: references/git.md.
  • ai-forge-eval: agents/refiner.md now states that it has not read the artifact, so every improvement must name a behaviour visible in the outputs — never structure, ordering, or prominence. A caller who knows the file discards the whole list when one item is invented.