Skip to content

gitmoot-skillopt v0.3.0

Choose a tag to compare

@jerryfane jerryfane released this 21 Jun 07:37
8dcefa2

gitmoot-skillopt v0.3.0

Ships judge-prompt optimization (#345 Phase 2) — the freeze-and-alternate counterpart to skill optimization. optimize can now tune the judge prompt against a held-out human-labeled set instead of the skill, gated on held-out human agreement.

Highlights

  • New optimize flags: --judge-prompt-optimization, --judge-human-labeled-path, --judge-prompt-init, --judge-prompt-version, --judge-edit-budget.
  • Runs a global pass plus one per task_kind; reflects a candidate judge prompt, scores its verdicts against the human-labeled set, and accepts only when held-out agreement improves (human_agreement gate). Judge verdicts are memoized by (prompt, item_id).
  • Emits a judge-candidate package (kind: gitmoot-skillopt-judge-candidate) with per-task_kind best prompt + version + agreement + gate history.
  • Validated on real data (live Codex judge): on a real held-out correctness set, agreement rose 0.333 → 0.667 (gate-accepted), with non-improving variants correctly rejected.

See docs/guide/judge-prompt-optimization.md and docs/reference/cli.md. Merged via #70 + #71.

Install

pipx install https://github.com/jerryfane/gitmoot-skillopt/releases/download/v0.3.0/gitmoot_skillopt-0.3.0-py3-none-any.whl