Skip to content

LOOP v0.3.1 - Reliability, Safety & Engineering-Quality Hardening

Latest

Choose a tag to compare

@killerdasher killerdasher released this 05 Nov 15:20

Reliability, safety, and engineering-quality hardening of the behavioral runner.

Security

  • Behavioral runner safety contract (tests/harness/run-case.mjs, new tests/harness/safety.mjs): an unattended evaluation no longer enables unrestricted automatic permission approval silently. A model run is now refused (exit 4, before any side effect) unless the operator acknowledges it explicitly with new --allow-auto; --build-only stays free of any acknowledgement. With acknowledgement, the runner (a) writes a least-privilege permission config to the evidence dir and delivers it through the documented OPENCODE_CONFIG tier — hard-denying external-directory access, webfetch, websearch, *.env reads, and curl/wget bash commands (explicit deny rules are the one thing OpenCode's auto mode does not approve) — and (b) builds the child environment from an allowlist (keeperEnv) so credential-shaped variables (API keys, *_TOKEN, SSH agents) are never inherited by the evaluated session — matched case-insensitively on Windows, whose real spellings are Path/ComSpec/SystemRoot (caught on a live machine: the original case-sensitive registry silently dropped ComSpec and SystemRoot from the evaluated session, degrading node/npm/PowerShell instead of merely scrubbing credentials), and case-sensitively on POSIX where path is a different variable; operator-named extras can be forwarded via --env-pass NAME,... (values never printed; the report records the count of dropped variables, never their names). Both A/B arms receive identical permission rules and identical env scrubbing. Documented honestly as containment, not an OS sandbox: bash/edit/read remain allow by default and the user's global OpenCode config still merges in (full isolation taxonomy now in tests/harness/README.md; SECURITY.md scope extended to the harness).

Fixed

  • Behavioral runner (unattended mode): the console summary printed the names of dropped credential-shaped environment variables while every report records only their count — contradicting the documented contract that names are never printed. Which variables exist on a machine is not public content, and runner console output often gets pasted into bug reports. The runner's console now shows the count-only summary via scrubSummary() (tests/harness/safety.mjs); operator-forwarded --env-pass names are unchanged, because the operator supplied those names. Regression-tested in the free self-test (safety:scrub-summary-count-only, safety:scrub-summary-empty, runner:scrub-summary-helper-in-use).
  • Behavioral harness on POSIX: runFixtureTest executed the fixture's command string with shell: false on non-Windows platforms, so spawnSync treated the whole string (node test/x.test.mjs) as a single filename — every seeded-state check failed with ERROR on ubuntu CI while Windows passed (first CI run to execute the harness self-test on Linux). Commands now run through a shell on every platform, with a pre-flight executable probe (where / command -v) so a missing executable stays an ERROR (broken environment) and can never be misread as a FAIL (red suite). Two free self-test checks guard both behaviors.
  • Installers (install.ps1, install.sh): timestamped backups were created in place — commands/loop.md.backup-<ts> and, for the skill, skills/loop-engine.backup-<ts> — but OpenCode discovers every directory containing a SKILL.md under skills/, so the overlaid backup came back as a phantom skill (the just-replaced version, live again). Observed with a real overwrite install on 2026-11-04. Backups now go to a sibling root (<dest>-loop-backups\<timestamp>\<tree>) outside all discovery directories, tree structure preserved; legacy in-place backups from older installers are left untouched. Installer tests assert the new location and that no *backup* residue remains inside commands/ or skills/.

For maintainers

  • Gates at this tag: scripts/validate.mjs --all 122/122 checks (79 structural + 27 mutation-injection checks that prove each new check can fail + 16 installer scenarios) and tests/harness/selftest.mjs 79/79 checks, both exit 0 on the tagged commit. CI (validate workflow, run 42059508351) passed on ubuntu-latest and windows-latest for this exact commit; CI never executes model sessions.
  • Tag re-anchor disclosure (for the record): this release was first published on 2026-11-05 from commit c83acb0 (self-test at 76/76 there, CI run 42053358009); the console-privacy fix listed under Fixed was folded into v0.3.1 the same day per maintainer decision instead of creating a v0.3.2, re-anchoring the annotated tag to the amended commit (eed4199). Main's history was appended, not rewritten (c83acb0 remains an ancestor); only the tag object moved, and only after both CI runners passed on the new commit. The re-anchor and the fix itself are disclosed here so the release page stays truthful about what the tag now contains.
  • History note (erratum, disclosed for the record): the installers fix was pushed on 2026-11-04 as commit 0b60dd7, whose message carried a false attribution clause ("Windows user report confirmed...") — the behaviors were tool-verified locally, and nothing external confirmed them — plus a non-existent GitHub runner name (linux-latest). Before the first publication, the message was corrected to "tool-verified locally, not a user report" and ubuntu-latest + windows-latest, and the corrected line was force-pushed as accadb4. git diff across that rewrite is empty: it changed no file content, only the message text of that one commit (its three descendants were re-set on top of it). The code itself was correct before; only the message was misleading.