Skip to content

Proctor v0.7.0

Choose a tag to compare

@github-actions github-actions released this 05 Apr 13:20
· 52 commits to main since this release

Verification contract gets two new gates

proctor record and proctor done now enforce an explicit intent-and-observation loop around every screenshot. The scenario description in the contract is no longer the only thing the agent has to produce — it also has to write what it's about to test (before capturing) and what it actually saw (after capturing).

The new loop

proctor start   → define happy / failure / edge-case scenarios
proctor note    → commit to intent BEFORE screenshotting
proctor record  → attach evidence (still does SHA binding + dedup)
proctor verify  → observation notes of what the screenshot actually shows
proctor done    → passes only if every scenario has note + record + verify

Gates that block the agent

  • proctor record refuses without a pre-note for the (scenario, session) pair
  • proctor done refuses while any scenario has pending-verification evidence
  • proctor done refuses if any scenario has no pre-note filed

Both pre-notes and post-verify observations have a 20-char minimum — enough to force specificity instead of "ok" or "tested it".

What's stored per run

  • notes.jsonl — append-only pre-test intent (audit trail, multiple notes per scenario allowed)
  • captures.jsonl — auto-populated ledger with one SHA entry per image artifact
  • evidence.jsonl — now carries status, notes, verified_at, and capture_ids linking back to ledger entries

Help text

proctor --help now opens with a showboat-style quickref: one-screen loop summary, command list, the three gates, minimum note length. Long-form teaching stays below the separator.

New commands

  • proctor note --scenario X --session Y --notes "..." — pre-test intent
  • proctor verify --scenario X --session Y --notes "..." — post-verify observation

Migration

Existing runs keep working — evidence without status is treated as pending, so old runs need one proctor verify pass per scenario to unblock proctor done.