perf(opencode): isolate per-proposal subprocess state to prevent SQLite WAL contention#35
Merged
Merged
Conversation
…te WAL contention OpenCode stores its session database at a global path (~/Library/Application Support/opencode/opencode.db on macOS, $XDG_DATA_HOME/opencode/opencode.db on Linux). When helix runs num_parallel_proposals > 1, all concurrent `opencode run` subprocesses issue `PRAGMA journal_mode = WAL` against this shared database at startup, producing: "Failed to run the query 'PRAGMA journal_mode = WAL'" This was observed in PR #34 E2E re-verify: g1-s1 lost to this SQLite WAL error while g1-s2 succeeded, wasting one proposal slot. Fix: set XDG_DATA_HOME to a per-candidate directory (<worktree>/.helix_opencode_state/) before invoking `opencode run`. OpenCode respects XDG_DATA_HOME and creates an isolated database there, so no two workers ever contend on the same file. - Only applies to `backend = "opencode"` (claude/codex/cursor/gemini untouched) - Skips the sandboxed path (container isolation already separates per-candidate state) - Adds .helix_opencode_state/ to .gitignore patterns (auto-cleanup with worktree) - 5 new unit tests covering: env set, uniqueness, env inheritance, backend isolation, gitignore; backend-exclusion test covers all 4 other backends (claude/codex/cursor/gemini) E2E validated: n_proposals=2 → 2/2 success, n_proposals=3 → 3/3 success, no SQLite WAL errors in either run. Cost: $0.00 (OpenCode free tier). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Owner
Author
|
Closing briefly to re-trigger CI after PR #34 merged into main. |
Owner
Author
|
Reopening; base auto-retargeted to main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
g1-s1failed with"Failed to run the query 'PRAGMA journal_mode = WAL'"whileg1-s2succeeded — wasting one proposal slotXDG_DATA_HOMEto a per-candidate directory (<worktree>/.helix_opencode_state/) before invokingopencode run, giving each parallel worker its own isolated SQLite databaseBackground
Depends on: #34 (perf/architecture-d-atomic-worker)
When
num_parallel_proposals > 1withbackend = "opencode", helix spawns concurrentopencode run --format jsonsubprocesses. All share the same global SQLite database:~/Library/Application Support/opencode/opencode.db$XDG_DATA_HOME/opencode/opencode.dbEach subprocess issues
PRAGMA journal_mode = WALat startup. Concurrent requests on the same file produce the error seen in PR #34's re-verify.Root cause archaeology: Confirmed via
opencode debug pathsthatXDG_DATA_HOMEchanges thedatadirectory (and thereforeopencode.dbpath). Setting it per-candidate gives complete isolation.Reference: PR #34 E2E re-verify report at
/Users/ke/helix-arch-d-e2e-reverify-report.md:OMAR comparison: OMAR uses
OPENCODE_CONFIG_CONTENTenv var for MCP config injection per-session, but does NOT setXDG_DATA_HOME. OMAR runs opencode in interactive TUI mode (tmux panes), not headlessopencode run, so it doesn't hit the cold-start WAL contention. TheXDG_DATA_HOMEisolation is helix-specific and implemented directly.Implementation
src/helix/mutator.py(~28 LOC including explanatory comment):Also adds
.helix_opencode_state/to_ignore_helix_artifactsgitignore patterns.E2E Validation Results
Both runs used the
add_oneoff-by-one fixture withbackend = "opencode",score_parser = "exitcode",frontier_type = "instance".n_proposals=2n_proposals=3Prior (PR #34 re-verify without this fix):
n_proposals=2→ 1/2 (g1-s1 SQLite WAL failure).Per-candidate isolation confirmed: each worktree has
.helix_opencode_state/opencode/opencode.db.Test Plan
uv run pytest tests/unit -q→ 871 passed (866 pre-existing + 5 new)uv run mypy --strict src/helix/→ Success: no issues found in 29 source filesn_proposals=2→ 2/2 success, no SQLite WAL errorsn_proposals=3→ 3/3 success, no SQLite WAL errors/Users/ke/helix-opencode-harden-review.md)🤖 Generated with Claude Code