LOOP v0.3.1 - Reliability, Safety & Engineering-Quality Hardening

Reliability, safety, and engineering-quality hardening of the behavioral runner.

Security

• Behavioral runner safety contract (tests/harness/run-case.mjs, new tests/harness/safety.mjs): an unattended evaluation no longer enables unrestricted automatic permission approval silently. A model run is now refused (exit 4, before any side effect) unless the operator acknowledges it explicitly with new --allow-auto; --build-only stays free of any acknowledgement. With acknowledgement, the runner (a) writes a least-privilege permission config to the evidence dir and delivers it through the documented OPENCODE_CONFIG tier — hard-denying external-directory access, webfetch, websearch, *.env reads, and curl/wget bash commands (explicit deny rules are the one thing OpenCode's auto mode does not approve) — and (b) builds the child environment from an allowlist (keeperEnv) so credential-shaped variables (API keys, *_TOKEN, SSH agents) are never inherited by the evaluated session — matched case-insensitively on Windows, whose real spellings are Path/ComSpec/SystemRoot (caught on a live machine: the original case-sensitive registry silently dropped ComSpec and SystemRoot from the evaluated session, degrading node/npm/PowerShell instead of merely scrubbing credentials), and case-sensitively on POSIX where path is a different variable; operator-named extras can be forwarded via --env-pass NAME,... (values never printed; the report records the count of dropped variables, never their names). Both A/B arms receive identical permission rules and identical env scrubbing. Documented honestly as containment, not an OS sandbox: bash/edit/read remain allow by default and the user's global OpenCode config still merges in (full isolation taxonomy now in tests/harness/README.md; SECURITY.md scope extended to the harness).

Fixed

• Behavioral runner (unattended mode): the console summary printed the names of dropped credential-shaped environment variables while every report records only their count — contradicting the documented contract that names are never printed. Which variables exist on a machine is not public content, and runner console output often gets pasted into bug reports. The runner's console now shows the count-only summary via scrubSummary() (tests/harness/safety.mjs); operator-forwarded --env-pass names are unchanged, because the operator supplied those names. Regression-tested in the free self-test (safety:scrub-summary-count-only, safety:scrub-summary-empty, runner:scrub-summary-helper-in-use).
• Behavioral harness on POSIX: runFixtureTest executed the fixture's command string with shell: false on non-Windows platforms, so spawnSync treated the whole string (node test/x.test.mjs) as a single filename — every seeded-state check failed with ERROR on ubuntu CI while Windows passed (first CI run to execute the harness self-test on Linux). Commands now run through a shell on every platform, with a pre-flight executable probe (where / command -v) so a missing executable stays an ERROR (broken environment) and can never be misread as a FAIL (red suite). Two free self-test checks guard both behaviors.
• Installers (install.ps1, install.sh): timestamped backups were created in place — commands/loop.md.backup-<ts> and, for the skill, skills/loop-engine.backup-<ts> — but OpenCode discovers every directory containing a SKILL.md under skills/, so the overlaid backup came back as a phantom skill (the just-replaced version, live again). Observed with a real overwrite install on 2026-11-04. Backups now go to a sibling root (<dest>-loop-backups\<timestamp>\<tree>) outside all discovery directories, tree structure preserved; legacy in-place backups from older installers are left untouched. Installer tests assert the new location and that no *backup* residue remains inside commands/ or skills/.

For maintainers

• Gates at this tag: scripts/validate.mjs --all 122/122 checks (79 structural + 27 mutation-injection checks that prove each new check can fail + 16 installer scenarios) and tests/harness/selftest.mjs 79/79 checks, both exit 0 on the tagged commit. CI (validate workflow, run 42059508351) passed on ubuntu-latest and windows-latest for this exact commit; CI never executes model sessions.
• Tag re-anchor disclosure (for the record): this release was first published on 2026-11-05 from commit c83acb0 (self-test at 76/76 there, CI run 42053358009); the console-privacy fix listed under Fixed was folded into v0.3.1 the same day per maintainer decision instead of creating a v0.3.2, re-anchoring the annotated tag to the amended commit (eed4199). Main's history was appended, not rewritten (c83acb0 remains an ancestor); only the tag object moved, and only after both CI runners passed on the new commit. The re-anchor and the fix itself are disclosed here so the release page stays truthful about what the tag now contains.
• History note (erratum, disclosed for the record): the installers fix was pushed on 2026-11-04 as commit 0b60dd7, whose message carried a false attribution clause ("Windows user report confirmed...") — the behaviors were tool-verified locally, and nothing external confirmed them — plus a non-existent GitHub runner name (linux-latest). Before the first publication, the message was corrected to "tool-verified locally, not a user report" and ubuntu-latest + windows-latest, and the corrected line was force-pushed as accadb4. git diff across that rewrite is empty: it changed no file content, only the message text of that one commit (its three descendants were re-set on top of it). The code itself was correct before; only the message was misleading.

LOOP v0.3.0 - Independent Engineering Audit & Behavioral Validation

Independent engineering audit and behavioral validation.

Added

• Behavioral evaluation harness (tests/harness/): semi-automated execution of evaluation Cases A–I — isolated fixture repositories (git-identity-isolated, temp-dir only), one model session per run through the officially documented opencode run --format json path, evidence collection (transcript, git diff, independent harness re-run of the fixture suite), and objective criteria with a four-level verdict (PASS / FAIL / BLOCKED / REVIEW, precedence FAIL > BLOCKED > REVIEW > PASS). Manual checkpoints return REVIEW and are adjudicated next to the evidence; the harness never auto-promotes them.
• Free harness self-test (tests/harness/selftest.mjs, run in CI): proves fixtures build, seeded states are real (failing fixtures fail, green fixtures pass, Case G's blocker blocks), criteria can both pass and fail, and the evidence parsers hold against the pilot-run defects. 64 checks, no model calls.
• Behavioral evaluation rubric rewrite (tests/evaluation.md): per case — controlled initial conditions, test objective, observable expected/forbidden behavior, evidence, and objective PASS/FAIL/BLOCKED/REVIEW criteria mapped to harness check IDs; honest result ledger (Case A pilot executed; B–I not yet run).
• A/B comparison methodology (tests/comparison.md): controlled protocol comparing standard OpenCode execution vs LOOP on identical tasks/snapshots/model/permissions, with metric definitions, anti-gaming reporting rules, and an explicit ledger stating no comparative results exist yet.
• ADR 0001 (docs/adr/0001-runtime-orchestration.md): compares skill-only, skill+plugin, and external-orchestrator architectures against official OpenCode APIs (verified); retains skill-only and records revisit triggers.
• Validator checks: evaluation-framework alignment (rubric vs in-session guidance must cover the same Cases A–I), changelog currency (a version beyond 0.1.0 must be documented, newest-first), reporting contract (iteration banner, SETTINGS: restatement, five-label status vocabulary), .gitignore must exclude behavioral evidence, CI must run the harness self-test — each with a mutation proving it can fail.

Fixed

• Harness pilot defects (found by executing Case A, not by inspection): phantom diff paths from trimming git status --porcelain output (src/range.mjs parsed as rc/range.mjs); the harness's own LOOP install polluting the scope check (arm setup is now committed to the fixture baseline without touching user work); read/skill/glob tool events misclassified as file writes (corrupted ordering facts); running/completed tool-event pairs double-counted.

Changed

• CI runs the free harness self-test in addition to the three validator modes.
• walkFiles in the validator excludes tests/harness/evidence/ (transient local run output is not repository content and must never fail repository checks).

LOOP v0.2.0 - Verified rebuild

Verified rebuild of LOOP. Same public interface as 0.1.0 (/loop command, loop-engine skill, file layout, flags), re-engineered and proven against OpenCode v2.2.5 with an end-to-end live-session run.

Added

• End-to-end verification in a live OpenCode v2.2.5 session: skill load, $ARGUMENTS flag parsing (SETTINGS: mode=... iterations=...), Stage 0-8 execution, LOOP | ITERATION n/N banner, and byte-level artifact confirmation of the produced file.
• stage: line in the iteration reporting banner (the 0.1.0 banner carried changed/verified/key findings/next only).
• --dry-run / -DryRun mode in both installers, with a test scenario proving it changes nothing.
• Backup-clobber regression scenario: a 60-second marker window proves an overwrite never destroys an earlier backup, for both installers.
• Secret-pattern scan in the validator (private-key blocks and GitHub/AWS/Slack/OpenAI-style token shapes).
• Flags-only invocation fallback: /loop --mode deep with no task text now uses the conversation objective.
• .gitattributes enforcing LF endings for *.sh, so a Windows clone cannot break install.sh.

Changed

• Installer test matrix: 16 scenarios (fresh install, idempotent rerun, overwrite refusal, overwrite-with-backup, clobber window, dry-run, project scope, uninstall), executed against temporary directories only for both installers.
• /loop command invokes the skill tool in a parameter-agnostic way (0.1.0 hard-coded id).
• Validator split into --self-test and --install-test modes with 11 defect-injection mutations that must each be detected; relative-link verification upgraded from plain existence checks (which hide case mismatches on Windows) to case-exact resolution.
• CI runs on actions/checkout@v7 and actions/setup-node@v7; both ubuntu-latest and windows-latest are green.
• Documentation rewritten; fixed the broken <owner-placeholder> release link from 0.1.0.

Validation

83 automated checks pass locally (54 structural, 13 self-test, 16 installer) and the full suite passes in CI on Linux and Windows. See tests/evaluation.md for the behavioral Cases A-I rubric.

LOOP v0.1.0 - Iterative Engineering Engine for OpenCode

LOOP v0.1.0 - first public release

Iterative Engineering Engine for OpenCode.

Highlights

• /loop command + loop-engine skill: 8-stage loop lifecycle, modes focused|balanced|deep, --max-iterations 1-10 (default 5), honest evidence labels (VERIFIED / IMPLEMENTED BUT NOT VERIFIED / BLOCKED / NOT APPLICABLE / FAILED)
• Portable installers: powershell 5.1+, pwsh (Linux/macOS), and POSIX bash; backup-before-replace with collision-safe names, KEPT protection of unowned files, forced uninstall escape hatch
• Validator (node scripts/validate.mjs): structure/frontmatter/reference/docs/link checks, plus --selftest sandbox coverage for both installers (fresh install, idempotency, backup-before-replace, backup collision safety, selective uninstall, KEPT protection, forced uninstall, XDG default destination, project-local destination) and defect injections that must be caught by the real validator binary
• CI: Node 18/22/24 across ubuntu and windows runners

Verification at release

• Local (Windows): validate.mjs 8 passed / 0 failed / 1 skipped; --selftest 17 passed / 0 failed / 1 skipped
• POSIX installer coverage additionally delegated to CI runs
• Honest labeling: the loop execution behavior itself is prompt-level and is therefore IMPLEMENTED BUT NOT VERIFIED; docs distinguish verified behavior from model-adherence expectations

See README.md for installation and usage.