Repository navigation
v3.16.4
What's new
One theme dominates this release: the OpenCode host contract stopped being a local ritual and became a required, fail-closed CI gate. A five-unit plan landed end to end (#916, #928, #930, #931, #937), and the very first real-host CI runs immediately caught four bugs sitting in already-merged code — each now pinned by a host-free regression test. In parallel, the typecheck:all backlog opened in #914 was burned all the way down: 343 errors across 30 files, to zero.
Bug Fixes
systematic_workflow_complete could report success while carrying stale evidence. The tool returns runtime.metadata() first, which may describe a ready-but-not-finalized unit (reasonCode: 'unit-ready', no workflowGuard block). Four branches in finishComplete()/finalizeComplete() returned without ever rewriting that output, so a completion the host marked completed was indistinguishable from a real success. Each branch now writes an explicit terminal result using the existing WorkflowReasonCode vocabulary — invalid-transition for a mismatched finalize target, guard-unavailable when nothing replayable is known, finalization-failed on an unavailable readback, and failed-operation when the host tool itself failed. No new status or reason literal was introduced, and the guard's state-machine semantics are unchanged; only the evidence written to tool output changed.
Two follow-on defects surfaced during review of the same change and were fixed in it. The completion writers replaced output.metadata wholesale instead of layering onto it, dropping the host's own failure sentinel on the shared output reference and letting a second guard instance take the success path. And metadata() emitted a bare, unprefixed status key whenever workflow_guard.debug was enabled, which still overwrote the host's status: 'error' sentinel even after the layering fix — reintroducing the same fail-open through a user-settable option. That debug-only key is now debugEpochStatus, so it cannot collide with a host-owned status field in any mode. Seven bidirectional proofs back the change (#945).
Build System
The pinned OpenCode development dependency moved to v1.18.22 (#880). Because the host pin is now derived from package.json rather than hand-copied constants (see Code Refactoring below), a bump like this is a single-field edit that the host-contract job re-proves automatically.
Documentation
Two solutions docs record what the host-contract work actually taught. host-contract-evidence-is-ci-owned-2026-09-04.md keeps the surviving invariant from the earlier 2026-08-16 doc — evidence gathered at one pinned OpenCode version is not evidence at another — and replaces the mechanism: the pin lives in package.json, the host is bunx opencode-ai@<pin>, and the evidence is the required host-contract job that release depends on. A Renovate OpenCode bump is now complete when that job is green on the PR; nobody re-runs the suite by hand. The superseded doc keeps its body as history with status: superseded frontmatter and a pointer to the replacement (#937).
The companion doc, real-host-ci-catches-what-offline-gates-cannot-2026-09-06.md, catalogues the bug classes only a real host can catch: four defects in merged code that passed unit tests, typecheck:all, lint, content-integrity, review, and offline skip runs, and were caught only by the new required job. It records the measured launcher topology on both macOS and Linux, two diagnostic traps, and the practice of pinning each host-caught bug with a host-free unit test (#938).
Tests
The real-host CI job found four bugs on its first runs, and this release fixes all of them.
A 180-second hang with zero output turned out to be stdio: spawnOpencodeChild used Node's default spawn stdio, leaving stdin an open, unclosed pipe that a real opencode run blocks on waiting for EOF. Switching to stdio: ['ignore', 'pipe', 'pipe'] took the previously-hanging tests from a 180s kill to a ~25s pass. The same PR converted one remaining bare opencode spawn site — missed by the earlier bunx conversion — to bunx opencode-ai@<pin> (#932).
The full integration suite exited 130 (SIGINT) even though every individual test passed. process is a single process-wide emitter shared by every test file in a Bun worker, so one eval-runner test's process.emit('SIGINT') reached the host fixture's never-removed SIGINT safety net, which called process.exit(130) unconditionally. The synthetic signal is now isolated to the listener under test (#933).
A test asserting that the fixture's launcher pid equals the pid the plugin reports from inside the OpenCode process passed on macOS and failed in CI with values exactly +1 apart. bunx execs the binary in place on macOS but forks it as a child on Linux, so the identity never held cross-platform. The test now asserts what it actually needs to prove — one probe load per host generation, in order (#936).
The fixture's reaper failed open: isProcessGroupAlive() treated every error from process.kill(-pid, 0) as proof the group was gone, collapsing POSIX EPERM ("exists, not yours to signal") into ESRCH ("gone"). That let the pruner drop a live entry and the SIGKILL sweep skip a still-running group, leaking a live opencode process past exit — a failure mode that previously caused a machine reboot on this project. It now returns false only for ESRCH and fails closed on everything else (#943).
A broader hardening pass on the host fixture's spawn, stdio, and signal handling landed alongside these: pruneLiveOpencodeProcesses() re-probes and evicts stale pids on every check instead of relying on a one-shot eviction at settle time, multi-byte UTF-8 chunk reassembly is pinned by a test, and buildDetachedChildSpawnOptions is now the single shared spawn-options builder both spawn sites call through, so they cannot diverge on stdio (#941).
Two host-free source-scan guards now pin the bug classes CI caught, so they cannot regress silently: one rejects any spawn call whose first argv element is the literal 'opencode', the other rejects any process.emit of a terminal signal anywhere under tests/ (#939).
The rest of the Tests bucket is the typecheck:all burn-down against #897, which finished in this release. The four highest-error test files accounted for 195 of the 343 baseline errors and went to zero by narrowing discriminated-union results behind explicit helpers, giving fixture builders real return types instead of Record<string, unknown>, and correcting a wrong-module type import (#919). Three real-host integration files adopted a per-file unwrapData() helper rather than reading the SDK's {data?, error?} response wrapper unguarded (#922). The eval fixture factories and case unions were typed against the real EvalResult, SourceProvenance, and CaseId exports (#923). Six unit files got complete SDK-shaped Model fixtures, ToolResult union narrowing, and type guards in place of runtime-only assertions (#924). The final batch took the count from 55 to zero, with the only source-side changes being additive: exporting CapabilityCliRoots, and adding a narrow read helper for the skill permission map that config-handler.ts genuinely writes but the imported v1 Config type omits (#926). No assertion behaviour changed anywhere in the burn-down.
Separately, the last two mkdtempSync cleanup gaps in the unit suite were closed — agent-resolver and pi-delegate-session were leaving 19 orphaned temp directories per run (#918).
Continuous Integration
The headline change is a required host-contract job that runs tests/integration against a real, bunx-launched OpenCode host and gates release. It builds dist/ before testing (load-bearing: the dist-local assertion is computed at module scope, so a late build would silently skip it), sets SYSTEMATIC_REQUIRE_OPENCODE=1 in workflow-owned env that a fork PR cannot alter, and always runs to completion so it is a valid required check. Path gating lives at step level and fails closed on any unexpected filter output. A JUnit-parsing guard then refuses to pass unless every one of the eleven integration files produced at least one test suite, the skipped set matches the known-exempt set exactly, and the pass count clears a floor — so "green with everything skipped" is not reachable (#931).
The wider type check landed as a staged rollout: typecheck:scripts covering src/, scripts/, and docs/scripts/ as a required step with no backlog, and typecheck:all covering tests/ too as advisory (continue-on-error: true) so the then-343-error backlog could surface without gating merges (#914).
The Node ESM load probe now covers dist/cli.js. package.json declares bin.systematic -> ./dist/cli.js, but the probe only imported dist/index.js and dist/pi.js — so a bare specifier that resolves under Bun but not plain Node would have shipped an unloadable systematic bin with no CI signal. This is not hypothetical: another project shipped exactly that failure from a bare jsonc-parser import resolving to a UMD main, and Systematic imports jsonc-parser bare in src/lib/setup.ts and src/lib/config.ts. The probe asserts loadability only, since dist/cli.js legitimately exports named symbols (#940).
Code Refactoring
The pinned OpenCode host version now has one readable source. scripts/lib/opencode-pin.ts reads the @opencode-ai/sdk devDependency from package.json, resolved relative to the module rather than cwd, and throws a clear error naming the field if it is missing or not an exact version. The two hand-maintained constants in scripts/run-evals.ts and the integration fixture became re-exports of it, retiring a four-way lockstep (#916).
Every host the suite spawns now launches through bunx opencode-ai@<pin> instead of npx, with no PATH lookup. A new probeOpencodeAvailability classifies a host as available, mismatch (exit 0 at the wrong version), or unavailable, and requireOpencodeAvailable turns a bad classification into a module-scope throw when SYSTEMATIC_REQUIRE_OPENCODE=1. Critically, the fixture module itself probes nothing at module scope, so tests/unit importers never spawn bunx. The shared bun install cache directory is resolved with lstat on every call — never stat, which would follow a planted symlink — and is refused outright if it is a symlink, not a directory, or not owned by the current uid (#928).
The host model test's last network dependency is gone. runOpencode now starts a local OpenAI-compatible scripted model server per invocation and merges its provider config into the fixture's isolated config, dispatching on request-body shape rather than call order — OpenCode forks a background title-generation call concurrently with the main chat loop, so ordering assumptions do not hold. All seven call sites were left untouched because the scripted-provider logic lives entirely inside runOpencode. The same change deleted the suite's only unfalsifiable test, which asserted that a cell status was one of two values that covered every possible outcome (#930).
Also in this release
- deps: update fro-bot/agent to v0.109.1, v0.109.2, v0.109.3, v0.109.4 (#917, #920, #921, #929)
- deps: update bfra-me/.github action to v4.26.0 (#934)
- dev: update dependency @biomejs/biome to v2.5.12 (#925)