Skip to content

v0.7.0: measured triggers, bench and injection scan

Choose a tag to compare

@github-actions github-actions released this 03 Nov 12:27
· 7 commits to main since this release
v0.7.0
6af58f4

v0.7.0: measured, not claimed

0.7.0 turns the Test phase into numbers: does the skill fire, does it help, and is a downloaded skill safe to install.

What changed in 0.7.0

  • Spec lint: frontmatter checked against the Agent Skills spec (allowed keys, compatibility, metadata, allowed-tools, name rules); descriptions that narrate steps are flagged
  • Injection scan: lint fails on hidden Unicode (zero-width, bidi), injection phrases and symlinks that leave the repo; warns on HTML comments, long base64 runs and scripts that read credentials
  • install_test.sh --npx: proves the npx skills add path (pinned skills@1.7.4) in a throwaway HOME
  • trigger_test.sh: measured recall and false-trigger rate over evals/triggers.json (billed, plan first)
  • bench.sh: with-vs-without benchmark, N runs per case, writes a dated Results block (billed, plan first)
  • badges.sh --from: a benefit badge prints only when its numbers come from a measured Results block
  • Lint catches multi-document case.yaml and invalid grader focus (both made cases silently skip)

Measured (2026-11-03)

  • Triggers: recall 0.95, 0 false triggers over 33 runs (holdout 0.83)
  • Bench, 5 cases x 3 runs: 47% of runs pass with the skill vs 0% without; score 0.78 vs 0.38; spend +53%, time -44% per run (evals/bench/2026-11-03.md)
  • Open: the injection-scan case passes 1/3 (the skill does not always fire on it); two cases not yet re-measured

Verify the zip
slsa-verifier verify-artifact skill-forger-e2e-v0.7.0.zip --provenance-path skill-forger-e2e-v0.7.0.intoto.jsonl --source-uri github.com/maumoya/skill-forger-e2e --source-tag v0.7.0

Full Changelog: v0.6.0...v0.7.0