Skip to content

opencode run (headless) never exits after a fatal non-retryable provider error — hung 3 days on "Monthly usage limit reached" #42268

Description

@DaisukeYoda

Summary

When the provider returns a fatal, non-retryable API error on the very first stream, opencode run (headless) logs the error and then never exits. The process stays alive indefinitely, produces no further stdout/stderr, and returns no exit code.

In my case three independent opencode run invocations hit AI_APICallError: Monthly usage limit reached about 3 seconds after start, and were still running 3 days later (78+ hours) when I killed them manually. They were not blocked on a dead socket — the runtime was alive the whole time, still firing its hourly snapshot-cleanup timer.

This is the headless counterpart of #32366 (TUI stuck on "thinking" after a stream error, no state recovery). For run the consequence is worse: there is no user watching, and no exit means any supervising wrapper is pinned open forever.

Environment

opencode 1.18.15 (npm i -g opencode-ai)
OS macOS 15.3.1 (24D70), Darwin 24.3.0, arm64
Hardware Apple M4, Mac mini
Node v24.5.0
Provider / model opencode-go / deepseek-v4-pro
Invocation non-interactive, via launchd (no TTY)

Command (run on a schedule, 3 different agents/skills):

opencode run --auto --dir /path/to/project --model opencode-go/deepseek-v4-pro "<prompt>"

What happened

Three scheduled runs started at 00:00, 04:00 and 08:00 UTC on 2026-08-10. All three:

  1. bootstrapped normally, created a session, entered loop step=0
  2. issued the first stream request
  3. received AI_APICallError: Monthly usage limit reached. Resets in 4 days.
  4. never exited — still alive 78 hours later

All three stopped emitting output at the identical point — the agent header line, with nothing after it:

> build · deepseek-v4-pro

(Same terminal symptom as the closed #17516, but a different trigger: that one hung after a successful run's tool calls, this one hangs on a failed first request.)

Evidence

From ~/.local/share/opencode/log/opencode.log (paths/IDs redacted). Run cdd5bd15, started 00:00:05Z:

timestamp=2026-08-10T00:00:05.517Z level=INFO  run=cdd5bd15 message="creating instance" directory=<project>
timestamp=2026-08-10T00:00:08.877Z level=INFO  run=cdd5bd15 message="llm runtime selected" llm.runtime=ai-sdk llm.provider=opencode-go llm.model=deepseek-v4-pro
timestamp=2026-08-10T00:00:09.430Z level=ERROR run=cdd5bd15 message="stream error" providerID=opencode-go modelID=deepseek-v4-pro
    session.id=<redacted> small=false agent=build mode=primary
    error.error="AI_APICallError: Monthly usage limit reached. Resets in 4 days. To continue using this model now,
    enable usage from your available balance: https://opencode.ai/workspace/<redacted>/go"
timestamp=2026-08-10T00:17:21.861Z level=ERROR run=cdd5bd15 message="stream error" ... small=true agent=title mode=primary
    error.error="AI_RetryError: Failed after 3 attempts. Last error: Cannot connect to API: The socket connection was closed unexpectedly."

After that second error at 00:17Z, the only thing this run ever logged again — for the next 3 days — was its hourly cleanup timer:

timestamp=2026-08-10T01:17:26.9Z  level=INFO run=cdd5bd15 message=cleanup prune=7.days
timestamp=2026-08-10T02:17:26.5Z  level=INFO run=cdd5bd15 message=cleanup prune=7.days
...
timestamp=2026-08-13T06:17:37.5Z  level=INFO run=cdd5bd15 message=cleanup prune=7.days   <- last, 78h after start

Of that run's 106 total log entries, 79 are this cleanup line. The other two runs are identical in shape (99 entries / 57 cleanup, 102 / 59).

The process was measurably spinning, not cleanly blocked:

$ ps -o pid,stat,etime,time -p 8231
  PID STAT   ELAPSED      TIME
 8231 S    03-06:53:07  55:13.03     # 55 min CPU over 78.9 h ≈ 1.2% sustained

Side effect worth noting: those hourly cleanups eventually started failing against each other, since three abandoned runs were all gc'ing the same snapshot dir —

level=WARN run=976bb9a4 message="cleanup failed" exitCode=128
    stderr="fatal: gc is already running on machine '<host>' pid 68550 (use --force if not)"
level=WARN run=20aebd72 message="cleanup failed" exitCode=128
    stderr="fatal: Unable to create '<snapshot>/....lock': File exists"

Expected behavior

A fatal, non-retryable provider error (quota exhausted, auth failure, 4xx that will never succeed on retry) in headless run should print the error to stderr and exit with a non-zero status.

Actual behavior

The error is written to the log only, nothing reaches stderr, and the process runs forever.

Impact

This is specifically painful for scheduled/CI use, which is what run is for. My jobs are wrapped in caffeinate -i (macOS: hold a sleep assertion for the child's lifetime). Because opencode run never exited:

  • 3 stuck processes held 78 hours of PreventUserIdleSystemSleep — the machine could not sleep for 3 days while nobody was using it, and ran hot the whole time
  • ~950 MB RSS retained
  • the scheduled job never ran again (launchd won't relaunch a label whose previous instance is still alive), so 3 days of audits were silently skipped with no failure signal anywhere

Any supervisor that waits on the child — CI runner, systemd unit, cron + wrapper, GitHub Actions step — has the same exposure. The only reliable mitigation today is an external wall-clock timeout, which I've since added.

Reproduction

Deterministic path I hit (3/3):

  1. Exhaust the monthly quota on an opencode-go model
  2. opencode run --auto --dir <project> --model opencode-go/deepseek-v4-pro "<any prompt>" with no TTY
  3. Observe stream error ... AI_APICallError: Monthly usage limit reached in the log, > build · <model> as the last stdout line, and a process that never returns

I'd expect any fatal non-retryable provider error to do the same (e.g. a revoked/invalid API key), but I have only verified the quota variant.

Suggested fix

In the headless run path, treat a stream error that is not retryable as terminal: surface it on stderr and exit non-zero, rather than falling through to an idle event loop. A belt-and-braces --timeout flag on run would also help, since it would bound any future hang class in unattended use.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions