Skip to content

Fix RunInTerminalTool hang when shell exits before/during execute()#313249

Merged
meganrogge merged 4 commits intomainfrom
fix/run-in-terminal-tool-already-exited
Apr 29, 2026
Merged

Fix RunInTerminalTool hang when shell exits before/during execute()#313249
meganrogge merged 4 commits intomainfrom
fix/run-in-terminal-tool-already-exited

Conversation

@meganrogge
Copy link
Copy Markdown
Collaborator

@meganrogge meganrogge commented Apr 29, 2026

Fixes #313248

Problem

In benchmark eval run 25073061392, 16 tests failed with X_AGENT_STILL_RESPONDING (60-minute outer timeout). All 16 traced to the same hang in RunInTerminalTool rich execute strategy. The same bug was independently confirmed in run 25115115244, where 13/89 tasks timed out with the identical pattern — shell process died (exit codes 1/2/22/127/130), onExit already fired, Event.toPromise hung forever.

Screenshot 2026-04-29 at 12 12 39 PM

The build under test already had #312827 (onExit listener) and #312854 (downgrade rich → basic on broken shell integration), so the listener wasn't missing. The actual bug was a subscription-ordering / already-fired-emitter race.

RichExecuteStrategy.execute() wired onExit / onDisposed after await this._instance.xtermReadyPromise. Two failure modes from this:

  1. Already-dead instance. If the pty had exited before execute() was entered, _onExit had already fired and been disposed (see terminalInstance._onProcessExit). Event.toPromise(onExit) then subscribed to a dead emitter and never resolved — hang until the agent's 60-min outer timeout.
  2. Race during xtermReadyPromise. If the pty exits during xterm init, the events fire before our subscription is attached and are missed.

BasicExecuteStrategy already wired its race before the await, but had the same upfront-already-dead hole.

Fix

In both richExecuteStrategy.ts and basicExecuteStrategy.ts:

  1. Synchronous early-out at the top of execute():
    • instance.isDisposed → throw "The terminal was closed" (matches existing onDisposed race branch).
    • instance.exitCode !== undefined → resolve immediately with the captured exit code and additionalInformation: 'Command exited with code N'.
  2. In rich, move the Promise.race([...]) lifecycle subscriptions before await xtermReadyPromise so onExit / onDisposed are wired synchronously at function entry. Closes the in-flight death race.

Tests

Added unit tests to both richExecuteStrategy.test.ts and basicExecuteStrategy.test.ts:

  • returns immediately with captured exit code when pty has already exited before execute()
  • throws "The terminal was closed" when instance is already disposed before execute()

runCommand / sendText are stubbed to throw, proving the early-out path doesn't try to write to a dead shell.

The rich execute strategy subscribed to onExit/onDisposed AFTER awaiting
xtermReadyPromise. If the pty had already exited before execute() was
entered, those emitters had already fired and been disposed, so
Event.toPromise() subscribed to a dead emitter and never resolved -
hanging the run-in-terminal tool until the agent's 60-minute outer
timeout (16 X_AGENT_STILL_RESPONDING failures observed in eval run
25073061392).

- Add synchronous up-front check for instance.isDisposed / exitCode in
  both rich and basic strategies; resolve immediately with the captured
  exit code rather than subscribing to a fired-and-disposed emitter.
- In the rich strategy, move the Promise.race lifecycle subscription
  setup BEFORE 'await this._instance.xtermReadyPromise' so onExit /
  onDisposed are wired synchronously at function entry, closing the
  race window where the pty exits during xterm initialization.
- Add unit tests for both branches in rich and basic strategies.

Fixes #313248
Copilot AI review requested due to automatic review settings April 29, 2026 15:09
@meganrogge meganrogge self-assigned this Apr 29, 2026
@meganrogge meganrogge added this to the 1.119.0 milestone Apr 29, 2026
@meganrogge meganrogge enabled auto-merge (squash) April 29, 2026 15:10
@meganrogge meganrogge added the ~requires-eval-assessment Evals will be run and will generate a report upon completion label Apr 29, 2026
@meganrogge meganrogge removed the ~requires-eval-assessment Evals will be run and will generate a report upon completion label Apr 29, 2026
@meganrogge
Copy link
Copy Markdown
Collaborator Author

/requires-eval-assessment terminalbench2 gpt-5.4,claude-opus-4.6,claude-opus-4.7

@vs-code-engineering
Copy link
Copy Markdown
Contributor

⏳ Queued vscode build for 05871ecad4d8b9be3ecbb92f5aee1aa85ec948f9 (step 1/2).

@vs-code-engineering vs-code-engineering Bot added the ~requires-eval-assessment Evals will be run and will generate a report upon completion label Apr 29, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a hang in the run_in_terminal tool’s rich/basic execute strategies when the terminal process has already exited (or exits during strategy setup), by ensuring lifecycle completion can’t be missed due to late event subscription.

Changes:

  • Add synchronous “already disposed / already exited” short-circuit handling at the start of execute() in both strategies.
  • In rich strategy, subscribe to lifecycle events before awaiting xtermReadyPromise to avoid missing onExit/onDisposed.
  • Add unit tests covering the “already exited” and “already disposed” early-out behavior for both strategies.
Show a summary per file
File Description
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/executeStrategy/richExecuteStrategy.ts Adds early-out checks and moves lifecycle subscriptions ahead of await xtermReadyPromise to prevent hangs.
src/vs/workbench/contrib/terminalContrib/chatAgentTools/browser/executeStrategy/basicExecuteStrategy.ts Adds early-out checks to avoid waiting on already-fired lifecycle events.
src/vs/workbench/contrib/terminalContrib/chatAgentTools/test/browser/richExecuteStrategy.test.ts Adds tests for immediate return on captured exit code and for disposed-instance rejection.
src/vs/workbench/contrib/terminalContrib/chatAgentTools/test/browser/basicExecuteStrategy.test.ts Adds analogous tests for the basic strategy.

Copilot's findings

  • Files reviewed: 4/4 changed files
  • Comments generated: 2

meganrogge and others added 2 commits April 29, 2026 11:44
…r/executeStrategy/richExecuteStrategy.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…r/executeStrategy/basicExecuteStrategy.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@vs-code-engineering
Copy link
Copy Markdown
Contributor

🚀 Queued eval-assessment publish build for ab48c2d6c59928e1a16d01cafce525816f064139 (step 2/2).

@vs-code-engineering
Copy link
Copy Markdown
Contributor

🔬 Queued eval-assessment benchmark for cd9772c658.

Results will be posted back here when the run completes.

@vs-code-engineering
Copy link
Copy Markdown
Contributor

✅ Eval-assessment build published.

@vs-code-engineering vs-code-engineering Bot removed the ~requires-eval-assessment Evals will be run and will generate a report upon completion label Apr 29, 2026
@meganrogge meganrogge merged commit cdbdcc3 into main Apr 29, 2026
26 checks passed
@meganrogge meganrogge deleted the fix/run-in-terminal-tool-already-exited branch April 29, 2026 16:19
@vs-code-engineering
Copy link
Copy Markdown
Contributor

📊 Eval-assessment benchmark complete.

Analysis Results

Resolution Rate

Benchmark Total Cases Passed Failed Resolved Rate
terminalbench2 89 56 33 62.92%

Token Usage

Metric Value
Total Tokens 79,996,111
Input Tokens 78,701,373
Output Tokens 1,294,738
Cached Tokens 75,538,682

Step Counts

Metric Value
Total Steps 1,798
Mean Steps/Instance 20.20

@vs-code-engineering
Copy link
Copy Markdown
Contributor

📊 Eval-assessment benchmark complete.

Analysis Results

Resolution Rate

Benchmark Total Cases Passed Failed Resolved Rate
terminalbench2 89 52 37 58.43%

Token Usage

Metric Value
Total Tokens 89,462,678
Input Tokens 88,259,039
Output Tokens 1,203,639
Cached Tokens 85,560,673

Step Counts

Metric Value
Total Steps 2,296
Mean Steps/Instance 25.80

@vs-code-engineering
Copy link
Copy Markdown
Contributor

📊 Eval-assessment benchmark complete.

Analysis Results

Resolution Rate

Benchmark Total Cases Passed Failed Resolved Rate
terminalbench2 89 48 41 53.93%

Token Usage

Metric Value
Total Tokens 50,061,498
Input Tokens 49,314,270
Output Tokens 747,228
Cached Tokens 42,203,392

Step Counts

Metric Value
Total Steps 1,323
Mean Steps/Instance 14.87

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RunInTerminalTool hangs for 60min when shell exits before/during execute() (rich strategy)

3 participants