Skip to content

v0.3.1 — default-on hard verifiers

Choose a tag to compare

@jerryfane jerryfane released this 23 Jun 05:06
b381dcc

v0.3.1 — default-on hard verifiers

Adds a default-on deterministic hard-verifier floor to the gitmoot evaluator (#346). evaluate_response now runs _run_hard_verifiers before the LLM judge and short-circuits with a hard=0 failure packet — shrinking the gameable surface and giving the optimizer crisp, actionable failures.

  • Built-in checks keyed by task_kind:
    • agent_template — valid YAML frontmatter, required "update format" section, fenced ```json blocks must parse, no secrets / absolute paths / remote-mutation·auto-promote language, size bounds.
    • package — valid JSON, strict contract_version == 1.
  • Honors declared evaluator_profile.checks; unknown task kinds with no declared checks are a no-op (behavior unchanged).
  • Fail-closed: every check runs under a guard that converts any crash (e.g. RecursionError from adversarial deeply-nested input) into a clean hard=0 failure instead of taking down the evaluator.

Verified: 403 pytest passing; reviewed (7 findings fixed); E2E with a real codex-generated agent-template (clean → judge; planted secret/abs-path → hard-failed before the judge).

Install: pipx install https://github.com/jerryfane/gitmoot-skillopt/releases/download/v0.3.1/gitmoot_skillopt-0.3.1-py3-none-any.whl