Skip to content

Releases: dev-geon/veriloop

v0.17.1 — Path-preserving artifact archive

Choose a tag to compare

@bgun42 bgun42 released this 29 Oct 02:01

Improvements

  • Archive worker artifacts under their staging-relative path. scripts/archive_run.py copied every artifact to artifacts/<worker>/<file name>, so two files with the same name in different staging folders (for example final/summary.json and red/summary.json) were refused with duplicate artifact basename. Artifacts now land at artifacts/<worker>/<staging-relative path>, and artifacts[] and checks[].artifact_path are rewritten to match. A repeated relative path, a path that differs only in case, and a path that leaves the staging directory are still refused before anything is written. Runs archived by earlier versions are unchanged.
  • Apply the single format correction to a plugin upgrade in the middle of a run: when a review or gate object returned earlier no longer validates against the new schema, the same role adds only what the new schema requires without changing its verdict or findings, and the controller never fills the missing fields in itself.

Verification

  • Contract suite passes with a new check for the mid-run version-change rule; removing the rule fails the suite.
  • evals/test_archive_run.py: 5 archives and 31 rejections (was 4 and 27), including two same-named artifacts in different folders archived with rewritten paths and identical bytes, and rejections for a repeated path, an aliased path (a/../a/...), a case-only difference, and a path escaping the staging directory. The symlinked-parent case is skipped where the platform denies symlink creation.
  • evals/test_review_report.py (5 valid, 32 rejections) and evals/test_prd.py (5 valid, 14 rejections) pass unchanged.

Full Changelog: v0.17.0...v0.17.1

v0.17.0 — Validated exit documents in Korean

Choose a tag to compare

@bgun42 bgun42 released this 27 Oct 12:32

Improvements

  • Write every document veriloop produces — IA.md, DECISIONS.md, REVIEW.md, PRD.md — in Korean, whatever language the conversation uses. Code, identifiers, file paths, commands, AC-<n> identifiers, verdict and severity tokens, and JSON blocks stay as they are. The IA.md and review report templates use Korean headings.
  • Validate PRD.md with the new scripts/check_prd.py: exactly ten fixed Korean ## chapters in order, a Korean body in each, a reason with every 해당 없음, and, with --spec, every AC-<n> criterion reported in 검증 with Pass, Fail, or Deferred; with --run-json, only after a goal_met exit. A failed validation is an archival failure.
  • Keep the ten chapter headings in the loop skill itself; the contract suite fails if that list, the exit-documents reference, and the validator disagree.
  • Move the REVIEW.md and PRD.md writing rules to skills/run-review-loop/references/exit-documents.md, read when a run ends.
  • Define the invariant oracle once in skills/draft-spec/references/invariant-oracle.md, referenced by both draft-spec and run-review-loop.
  • Remove repeated instructions: the inline gate JSON example, hand-written archive steps, the reviewer oracle checklist, per-object format-retry rules (now one rule for review and gate objects), and duplicated design-partner and model-group text. Re-dispatch the design partner only when an answer invalidates its implementation design.
  • run-review-loop/SKILL.md shrinks from 38.0 KB to 31.4 KB and draft-spec/SKILL.md from 15.7 KB to 15.1 KB. No measured token saving is claimed.

Verification

  • Contract suite passes, including the PRD chapter-list agreement check and the Korean document rules; mutating the chapter list in any of the three places, or removing the Korean rule, fails the suite.
  • New evals/test_prd.py: 5 valid documents passed and 14 invalid cases rejected with their expected reasons, including Korean particles attached to criterion identifiers.
  • evals/test_archive_run.py (4 archives, 27 rejections), evals/test_review_report.py (5 valid, 32 rejections), and evals/test_worker_result_cli.py (2 valid, 9 rejections) pass unchanged.

Full Changelog: v0.16.0...v0.17.0

v0.16.0 — Requirement-quality rules for specifications

Choose a tag to compare

@bgun42 bgun42 released this 27 Oct 12:32

Improvements

  • Hold specification statements to a requirement-quality reference (skills/draft-spec/references/requirement-quality.md), adapted from the requirement characteristics in ISO/IEC/IEEE 29148 and written as veriloop's own text so Codex and Claude Code apply the same rules.
  • Give every acceptance criterion a permanent AC-<n> identifier that is never renumbered or reused, and state one observable outcome per criterion. Run records and the PRD.md verification chapter cite criteria by identifier.
  • Replace words that leave a threshold open ("fast", "etc.", "if possible", "improved") with a bound, a complete set, or a reference behavior; a replacement that needs an unsettled value goes back to the user as a decision.
  • Add a non-functional lens (performance and cost bounds, permissions, reliability under failure) to decision-surface enumeration, applied only where the work touches it.
  • Add a third guess-hunt question: contradictions between sections, and criteria that join outcomes or leave their threshold open in a way that would let two readers build or judge different behavior.
  • Criterion priority is not part of this release.

Verification

  • Contract suite passes with four new draft-spec instruction contracts; removing any of the guarded sentences fails the suite.
  • evals/test_archive_run.py, evals/test_review_report.py, and evals/test_worker_result_cli.py pass unchanged.

Full Changelog: v0.15.0...v0.16.0

v0.15.0 — Evidence-grounded repair designs

Choose a tag to compare

@bgun42 bgun42 released this 22 Oct 00:44

Full Changelog: v0.13.0...v0.15.0

Veriloop 0.13.0

Choose a tag to compare

@bgun42 bgun42 released this 19 Oct 03:36

Improvements

  • Escalate a recurring invariant to an oracle: when one invariant fails in two iterations or causes no_progress/iteration_cap, stop symptom-by-symptom repair until the specification holds a confirmed invariant oracle (reference correct by construction, seeded generator, normalization, budgets, and strength switches that rediscover every historical failure). Reviewers and gates judge the oracle from the target itself.
  • Keep IA.md a contract: decisions and revision history move to a sibling DECISIONS.md, which reviewers and gates do not read; specs and code comments never cite finding identifiers or run numbers.
  • Preflight acceptance checks in draft-spec before confirmation, and give every criterion a phase (loop, post_deploy, manual). Only loop criteria decide goal_met; the rest are archived as deferred and tracked as open items.
  • Handle stalled workers (stall_minutes, default 20): the run ends fail-closed as worker_blocked, and an authorized successor continues only in a follow-up run (preceding_runs, worker_successions). Pre-authorized fallback_models substitute automatically and are recorded.
  • Issue unique per-attempt temporary paths to every role, and archive runs with scripts/archive_run.py (digest-verified artifact copies, review JSON checked against iterations, fail-closed exit rules, atomic move; exit 1 = nothing archived, exit 3 = archived but staging cleanup failed).
  • Define the REVIEW.md layout (current state, one delta section per run, one closing json veriloop-report block) and validate it with scripts/check_review_report.py against the new schemas/review-report.schema.json.
  • All new run.json fields are optional; existing archives remain valid. English and Korean READMEs and docs/ci.md document the changes.

Verification

  • Contract suite: existing counts unchanged (3 traces, reduced-mode transition, 3 worker exits, 24 guards, 7/21 worker results, 6 fallback paths); new phase/oracle/succession extensions: 3 passed, 7 guard mutations rejected; 15 new instruction contracts checked in run-review-loop and draft-spec.
  • evals/test_archive_run.py: 4 archives passed, 27 invalid cases rejected with their expected reasons. evals/test_review_report.py: 5 valid documents passed, 32 invalid cases rejected.
  • 28 real-world run archives, 18 gate results and 52 worker results from earlier loops still validate against the updated schemas.
  • One strict blind review: one Failed finding (same-run succession contradicted the fail-closed worker exit) and eight warnings fixed; one warning accepted and documented in docs/ci.md (the trace evaluator models recurrence by finding identity, not by invariant).
  • Manifest and schema JSON parse checks and git diff --check passed. The symlink case of evals/test_worker_result_cli.py needs symlink privilege and was not run on the Windows release host.

Strict blind isolation, the three-iteration cap, worker-result limits and late-bound gates remain unchanged. Python tools require Python 3.10+ and the standard library only.

Veriloop 0.12.1

Choose a tag to compare

@bgun42 bgun42 released this 19 Oct 00:31

Improvements

  • Group related findings into bounded root-cause repairs and verify preserved normal behavior.
  • Resolve loop vs standalone output/persistence responsibilities: workers return structured results; controller owns loop records.
  • Keep review histories and worker artifacts out of strict reviewer/gate reads, searches and diffs.
  • Reduce redundant checks and document rewrites while retaining independent controller checks, full final acceptance and required device verification before gate.
  • Require a changed approach for user-authorized follow-up runs after no progress; preserve the three-iteration cap.
  • Add a dependency-free Python worker-result validation CLI and regression tests.
  • Document the improvements and CLI usage in English and Korean README files.

Verification

  • Existing contract suite: 3 traces, authorized reduced-mode transition, worker exits, adversarial guards, worker schema/semantics and JSON fallback checks passed.
  • Runtime CLI: 2 valid cases passed; 9 invalid cases rejected.
  • Manifest JSON and skill metadata checks, git diff whitespace check passed.

Exact role models/efforts, fresh independent strict reviewers and gates, snapshot invalidation and required acceptance checks remain enforced. Python CLI requires Python 3.10+.

Veriloop 0.12.0

Choose a tag to compare

@bgun42 bgun42 released this 07 Sep 00:09

Three artifacts, three jobs

The loop now writes IA.md, REVIEW.md and PRD.md, and each does something the other two do not.

  • IA.md is the contract — the confirmed specification, every clause paired with a check that runs. It replaces the former SPEC.md.
  • REVIEW.md is the audit — the final review report, persisted beside the specification. apply-review-findings locates it by convention and syncs finding statuses back into it after fixes.
  • PRD.md is the product document — a description of what now exists, written for someone who was not in the room.

PRD.md is a product document, not a delivery checklist

The previous definition put the acceptance-criteria table at the centre. In use that produced a document nobody outside the work could read: verdicts were listed, but what the thing is, who uses it and how it works were stated nowhere. A verification table only means something when a product description stands in front of it.

PRD.md now carries ten chapters in order: what it is, who uses it, what problem it solves, what it does, how it is used, how it works, non-functional requirements, verification, what remains, and the document's own limits. Verification is one chapter, not the whole document.

Rules that come with the definition:

  • Chapters scale with the size of the work — a single bug fix gets a paragraph each and a complete verification table; a programme gets the whole document. No chapter is dropped to save space; "not applicable" is written down with the reason.
  • Factual claims are grounded in file:line. Inferences are marked as inferences.
  • Unresolved discrepancies are recorded as discrepancies, not quietly dropped.
  • Findings themselves stay in REVIEW.md; PRD.md references them.
  • Written on a goal_met exit only; otherwise the report says why it was not.

For a programme made of several pieces of work, the same pair scales to two tiers. IA.md and PRD.md in the parent directory describe the whole; each piece keeps its own pair in its own directory. Stakeholders read the parent PRD.md first.

Also in this release

  • The loop-dashboard skill is removed, along with its stale mentions in both READMEs.
  • draft-spec names the confirmed specification inside a per-work directory; the directory layout itself stays repository- or locally-governed.
  • Both READMEs now spell out what belongs in each artifact.
  • The Codex manifest, left at 0.10.0 through the 0.11.0 and 0.12.0 bumps, is back in step.

This release folds in the unreleased 0.11.0. The Claude and Codex marketplace manifests both publish version 0.12.0.

Veriloop 0.10.0

Choose a tag to compare

@bgun42 bgun42 released this 15 Aug 14:14

Bounded worker output and verified artifacts

  • Caps complete worker result envelopes at 16 KiB.
  • Limits evidence excerpts, tool errors, and blocker messages to 512 characters.
  • Stores essential long logs as isolated artifacts instead of adding agents or polluting controller context.
  • Verifies artifact paths, byte counts, SHA-256 digests, references, and size limits.
  • Keeps successful-path artifacts out of reviewer and gate context.
  • Expands deterministic worker-result and raw-JSON fallback guards.

Both Codex and Claude marketplace manifests publish version 0.10.0.

Veriloop 0.9.0

Choose a tag to compare

@bgun42 bgun42 released this 15 Aug 13:44

Worker completion protocol

  • Adds schema-constrained developer and fixer result envelopes.
  • Treats host timeout, cancellation, context limits, and tool failures as authoritative fail-closed exits.
  • Binds developer checks to the current acceptance batch and fixer checks to Failed findings.
  • Rejects mixed prose/JSON fallback responses and permits only one format-only retry.
  • Adds deterministic worker exit, contradiction, substitution, and fallback contract coverage.

Both Codex and Claude marketplace manifests now publish version 0.9.0.

Veriloop 0.8.0

Choose a tag to compare

@bgun42 bgun42 released this 14 Aug 10:51

Veriloop 0.8.0 renames the core review skill to $veriloop and makes strict blind review the default. Every iteration uses a unique reviewer, final gates generate late-bound holdout probes against a frozen snapshot, and reduced-independence review now requires explicit one-run authorization when clean subagent contexts are unavailable.