fix(e2e): stop paying opencode's per-directory plugin install ~40 times a run - #2270
Merged
Conversation
The opencode leg went red on 2026-09-03 and every failure looked the same: `agent failed: signal: killed` at the 120s prompt budget, with empty stdout and empty stderr. opencode prints nothing while it stalls during startup, so the artifacts named no cause — the leg was undiagnosable after the fact, and the elimination had to be done from timing alone (the CLI's own hooks were fine, and faster than in the last green run, so the added cost is inside opencode's startup). Fold opencode's own log directory into the artifact tree so the next occurrence carries the stalled call with it. Best-effort: a leg that never got as far as writing logs must not fail the step. Also raise the warmup budget from 30s to 90s and report each attempt's duration. The warmup exists to pay opencode's first-run costs once and serially before ~40 tests start in parallel; at 30s it was being killed part-way through on every run since the step change, leaving that work half-done for every test that followed. Reporting the duration makes the one serial, uncontended measurement of opencode startup visible in the CI log instead of only in a downloaded artifact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01M1NZ1KYM0K1S73B1EHB9Z9KK
…m per repo Whenever a project directory contains a local plugin file — which `entire enable` always writes — opencode generates `.opencode/package.json`, pinning `@opencode-ai/plugin` to its own binary version, and installs it (27 packages, ~61MB) before it will finish bootstrapping. The install runs inside a phase that logs nothing and has no timeout, so a slow one is indistinguishable from a hang: the process sits between "loading path=..." and "all LSPs are disabled" and produces no output at all. Every test gets a fresh repo, so that install was being paid ~40 times per run, in parallel. Since 2026-09-03 it stopped fitting in the budgets: in run 33862929010, 27 of 53 opencode processes died inside that window. Measured here, an unseeded directory takes 5m00.9s at 1% CPU to complete an install that takes 3s when npm is invoked directly. Build the tree once in Bootstrap, keyed by opencode version (the pin follows the binary, so an upgrade invalidates it), and plant it in each repo from a new RepoSeeder hook on SetupRepo. node_modules is symlinked rather than copied: at ~61MB across thousands of small files and ~40 repos live at once, copying would cost about what it saves. The seed also writes the same `.gitignore` opencode writes, which names its own file, so nothing we plant is visible to git — these repos are the subject of checkpoint assertions. Measured with a seeded directory: bootstrap reaches the model in 839ms. Every step degrades to today's behaviour rather than failing: no npm, an unreadable version, or a platform without unprivileged symlinks all leave the repo alone and let opencode install for itself. Note this is not a test-only workaround. Every user running opencode pays the same install on first launch in each repo, and again after each opencode upgrade, with a blank screen and no explanation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01M1P10839YG3R5AF274XYE6TT
The pre-seed landed but did nothing for the tests. Bootstrap runs in `go run ./e2e/bootstrap` and SetupRepo in `go test ./e2e/tests` — two processes — so the package variable Bootstrap set was always empty where SeedRepo read it. Seeding degrades quietly by design, so the failure looked like success: run 33865617639 reported deps built in 4.9s and a warmup that finally passed on the first attempt, while 31 of 53 opencode processes still died inside the install window. Resolve the tree through a sync.OnceValues instead, so either process finds it on disk or builds it. It lives under os.TempDir() rather than os.UserCacheDir() for two reasons: the e2e TestMain points XDG_CACHE_HOME at the artifact directory, so a tree built there would both miss the one Bootstrap built and be uploaded as a ~61MB CI artifact. The install is staged and renamed into place, so a reader never sees a half-installed tree and a process that loses the race adopts the winner's. The new test runs SeedRepo from a process that never called Bootstrap, which is the shape of the bug — the previous version passed every unit test and every local check while seeding nothing. Also narrows the claim in Bootstrap's comment to what the opencode source actually does: it forks the install for every existing config directory and blocks on all of them whenever any plugin is configured, so a repo pays only once it has both a project-local .opencode directory and a plugin configured anywhere. `entire enable` supplies both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01M1P2Q273Z8W236F5RE4GV00B
Cleanup pass over the preceding two commits; no intended behaviour change beyond the warmup budget and the warmup dir now getting the same config the test repos get. - The warmup went through a hand-rolled exec that had become a copy of RunPrompt once it needed cmd.Dir and openCodePromptEnv. Call RunPrompt. That env helper exists because opencode resolves its project root from PWD, which cmd.Dir does not set, so a divergence there warms the wrong directory silently — one call site is the point. - Retry at 30s rather than 90s. Only the first attempt can be paying opencode's real first-run costs, so 3x90s was over four minutes of blocked CI on the exact path the warmup exists to survive. - SetupRepo wrote opencode.json from an agent.Name() switch ten lines above the new RepoSeeder call, leaving two homes for one agent's per-repo setup. Moved into SeedRepo, ahead of the dependency-tree resolution so a repo still gets a usable config when the pre-build failed. - Use the package's build-tagged linkFile instead of a raw os.Symlink plus a comment restating what its Windows variant already documents. - Drop Bootstrap's deps if/else: buildPluginDeps reports the build and SeedRepo reports a failure, so it only added a third message about one event. Skipped from the review, with reasons: memoizing package.json and the lockfile alongside the path (measured under a millisecond across all ~40 repos, and it trades a plain path for a struct); pruning the collected opencode log dir (estimated at 15-50MB, measured at 464KB); moving log collection from the workflow into TestMain and documenting the install in the agent guide (both real, both outside this diff). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01M1P4SARPGWY5DMEJV74TNP4T
jdx
previously approved these changes
Sep 4, 2026
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The new seeding test currently assumes symlinked node_modules unconditionally and should be excluded from Windows builds to avoid expected failures.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR fixes recurring e2e-tests (opencode) timeouts by avoiding Opencode’s per-repo plugin dependency install during startup, while also improving diagnosability and making warmup behavior more observable.
Changes:
- Introduces an
agents.RepoSeederhook and updates repo setup to call it afterentire enable, allowing agent-specific pre-run seeding. - Implements Opencode repo seeding + once-per-process dependency prebuild (via
npm install) and updates Opencode warmup budgeting/logging. - Collects Opencode’s own logs into CI artifacts for post-failure diagnosis.
File summaries
| File | Description |
|---|---|
e2e/testutil/repo.go |
Calls an optional RepoSeeder after enabling Entire, so agents can plant required files once per fresh repo. |
e2e/agents/agent.go |
Adds the RepoSeeder interface and documents its best-effort contract. |
e2e/agents/opencode.go |
Adds Opencode dependency prebuild + repo seeding and improves warmup budgets/timing output. |
e2e/agents/opencode_seed_test.go |
Adds a regression test to ensure seeding works from the test process (not relying on bootstrap-only state). |
.github/workflows/e2e.yml |
Copies Opencode’s log directory into the artifact tree on the opencode CI leg. |
Review details
- Files reviewed: 5/5 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…n Windows Two review findings. sync.OnceValues cached a failure exactly as permanently as a success, so one transient npm error on the very first call would disable seeding for every repo in the run — reinstating the concurrent-install storm this change exists to prevent, for the whole run instead of one attempt, with only a stderr line as evidence. A success is still cached for the life of the process; a failure is now retried, capped at two attempts total so the opposite failure (~40 installs when npm is simply absent) stays out of reach. The lock is held across the build either way, so attempts are serial. The seed test asserted a symlinked node_modules unconditionally, which cannot hold on Windows: linkFile copies there, a copy cannot be a 61MB directory, and SeedRepo takes its documented degradation path. Rather than excluding the file from Windows builds — which would also drop the platform-independent assertions, including the cross-process resolution the test exists to pin — it now asserts the degradation on Windows and the symlink elsewhere. Not a live CI failure: the Windows job runs only ./cmd/entire/cli/... filtered to (Windows|MSYS), so this was reachable from a local Windows run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 01M1P74VTN4TAQXTCD7K87HJ1Q
gtrrz-victor
approved these changes
Sep 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://entire.io/gh/entireio/cli/trails/1238
What
The
e2e-tests (opencode)leg went red on main from 2026-09-03 22:10 UTC and stayed red. This makes it green again: run 33867981097 — all 56 tests passed,Run E2E Tests469s (it was 1266s and failing).Why it was failing
Every failure was
agent failed: signal: killedat the 120s prompt budget, with zero assertion failures and completely empty stdout and stderr. Nothing in the artifacts named a cause, because opencode prints nothing while it stalls.It was not a gradual degradation. The single-process bootstrap warmup was 9–15s for all 32 preceding runs, then 40s+ on every run after. Not ours, and not opencode's code either: same opencode version (1.18.27) on both sides of the boundary, same runner image, only trail-routing commits in our repo, the claude-code leg sharing the same API key got faster, our own hook perf was better in the red runs, and opencode's install path has one commit since Aug 25 (on 08-27).
What changed is that a cost which was always pathological stopped fitting in the budgets.
The mechanism
From opencode's source at 1.18.27:
ConfigPaths.directoriesreturns the global config dir plus every existing.opencodedirectory walking up from cwd.config.ts:452forks a background@opencode-ai/plugininstall for each of them.plugin/index.ts:184—if (plugins.length) yield* config.waitForDependencies()— then blocks on all of them, andwaitForDependenciesjoins the fibers with no timeout and logs nothing.So a repo pays a 27-package, ~61MB install once it has both a project-local
.opencodedirectory and a plugin configured anywhere, global or project.entire enablesupplies both by writing.opencode/plugins/entire.ts. Every test gets a fresh repo, so we were paying it ~40 times in parallel.It is also much worse than an npm install sounds:
npm.ts:81runs@npmcli/arborist'sreify()in-process rather than the npm CLI. Measured on one machine, identical 27-package result: arborist 5m00.9s at 1% CPU,npm install3s.What this does
RepoSeederhook onSetupRepo.node_modulesis symlinked, not copied — 61MB across thousands of small files times ~40 live repos would cost about what it saves. The seed also writes opencode's own.gitignore, which names its own file, so nothing planted is visible to git; these repos are the subject of checkpoint assertions.Every step degrades to today's behaviour instead of failing: no npm, an unparseable version, or a platform without unprivileged symlinks all leave the repo alone and let opencode install for itself.
Verification
Don't read the pass count alone — parse the newly-collected
opencode-logs/opencode.logfor the gap between the lastloading path=line and the next event. That is the install window.The 6 remaining are processes ended by test teardown, not stalls — with a 0.35s max gap there is nothing left to stall on.
Worth knowing
One intermediate revision here looked fixed and was not:
Bootstrapruns ingo run ./e2e/bootstrapandSetupRepoingo test ./e2e/tests, two processes, so handing the tree between them via a package variable seeded nothing. Because seeding degrades quietly, the bootstrap log reported success while 31 of 53 processes still stalled. The tree is now resolved per-process, andTestOpenCodeSeedRepoPlantsDepsrunsSeedRepofrom a process that never calledBootstrap— the shape of that bug.Not in scope
The same install hits real users under the same two conditions. Moving Entire's plugin to
~/.config/opencode/plugins/would make it a one-time global cost instead of per-repo, at the price of loading in every project including ones where Entire was never enabled — a design call, not a refactor. Upstream: anomalyco/opencode#47212.🤖 Generated with Claude Code
Note
Medium Risk
Touches only E2E/CI paths but adds cross-process caching, npm installs, and symlink semantics where quiet fallback could mask regressions.
Overview
Fixes opencode E2E timeouts caused by ~40 parallel, silent
@opencode-ai/plugininstalls (~61MB each) afterentire enableadds a project plugin.Adds a
RepoSeederhook so agents can plant files post-entire enable; opencodeSeedReponow writesopencode.json, copies pinnedpackage.json/ lockfile into.opencode, symlinks a sharednode_modulestree built once per process vianpm install(version-keyed underos.TempDir()), and mirrors opencode’s.gitignoreso seeded blobs don’t affect checkpoint/git assertions.Bootstrapwarms up in a seeded scratch dir throughRunPromptwith longer first-attempt budgets and per-attempt timing logs.CI copies
~/.local/share/opencode/loginto artifacts on the opencode matrix leg.TestOpenCodeSeedRepoPlantsDepsasserts seeding works from the test process without relying on bootstrap-only state.OpenCode-specific setup was removed from
SetupRepoin favor of the seeder interface; failures to pre-build or link deps degrade to opencode’s own install rather than failing setup.Reviewed by Cursor Bugbot for commit b055dcb. Configure here.