Repository navigation
v3.8.0
What's new
Features
Added a local OpenCode outcome runner for evals (#778), giving the project a deterministic way to execute and verify evaluation cases against local OpenCode runs without any telemetry or upload. The runner covers two cases — bootstrap-loading and fixture-local-write — each executable in both source and installed (packed-installed) modes, with distinct provenance tracked per mode so results are traceable back to how the code under test was actually running.
Fixture isolation is scoped rather than OS-sandboxed: child-process environment inheritance is denied by default, with explicit allowlisting required for test execution. Fail-closed behavior is enforced throughout — archive and dependency-resolution handling, privacy redaction, residue/quarantine cleanup, and manifest persistence all default to blocking rather than silently succeeding on the unsafe path. Signal-safe lifecycle handling was also wired in to guarantee best-effort cleanup of fixtures, spawned processes, and installed artifacts if a run is interrupted mid-flight, and outputs are written only to local run artifacts under evals/runs/<runId>/.
Follow-up review findings were folded into the same change: eval child-environment inspection was made CodeQL-safe by passing dynamic fixture values through argv instead of the environment, OpenCode secret matching was narrowed from a broad sk-* token regex to explicit sk- / sk_ shapes to cut false positives, and a regression test was added confirming legitimate skill-loading remains allowed while token-shaped sk values are still rejected.
Also in this release
- deps: Renovate updated
@biomejs/biometo v2.5.8 (#777).