Skip to content

Veriloop 0.13.0

Choose a tag to compare

@bgun42 bgun42 released this 19 Oct 03:36
· 9 commits to main since this release

Improvements

  • Escalate a recurring invariant to an oracle: when one invariant fails in two iterations or causes no_progress/iteration_cap, stop symptom-by-symptom repair until the specification holds a confirmed invariant oracle (reference correct by construction, seeded generator, normalization, budgets, and strength switches that rediscover every historical failure). Reviewers and gates judge the oracle from the target itself.
  • Keep IA.md a contract: decisions and revision history move to a sibling DECISIONS.md, which reviewers and gates do not read; specs and code comments never cite finding identifiers or run numbers.
  • Preflight acceptance checks in draft-spec before confirmation, and give every criterion a phase (loop, post_deploy, manual). Only loop criteria decide goal_met; the rest are archived as deferred and tracked as open items.
  • Handle stalled workers (stall_minutes, default 20): the run ends fail-closed as worker_blocked, and an authorized successor continues only in a follow-up run (preceding_runs, worker_successions). Pre-authorized fallback_models substitute automatically and are recorded.
  • Issue unique per-attempt temporary paths to every role, and archive runs with scripts/archive_run.py (digest-verified artifact copies, review JSON checked against iterations, fail-closed exit rules, atomic move; exit 1 = nothing archived, exit 3 = archived but staging cleanup failed).
  • Define the REVIEW.md layout (current state, one delta section per run, one closing json veriloop-report block) and validate it with scripts/check_review_report.py against the new schemas/review-report.schema.json.
  • All new run.json fields are optional; existing archives remain valid. English and Korean READMEs and docs/ci.md document the changes.

Verification

  • Contract suite: existing counts unchanged (3 traces, reduced-mode transition, 3 worker exits, 24 guards, 7/21 worker results, 6 fallback paths); new phase/oracle/succession extensions: 3 passed, 7 guard mutations rejected; 15 new instruction contracts checked in run-review-loop and draft-spec.
  • evals/test_archive_run.py: 4 archives passed, 27 invalid cases rejected with their expected reasons. evals/test_review_report.py: 5 valid documents passed, 32 invalid cases rejected.
  • 28 real-world run archives, 18 gate results and 52 worker results from earlier loops still validate against the updated schemas.
  • One strict blind review: one Failed finding (same-run succession contradicted the fail-closed worker exit) and eight warnings fixed; one warning accepted and documented in docs/ci.md (the trace evaluator models recurrence by finding identity, not by invariant).
  • Manifest and schema JSON parse checks and git diff --check passed. The symlink case of evals/test_worker_result_cli.py needs symlink privilege and was not run on the Windows release host.

Strict blind isolation, the three-iteration cap, worker-result limits and late-bound gates remain unchanged. Python tools require Python 3.10+ and the standard library only.