Skip to content

Conversation

@ammar-agent
Copy link
Collaborator

Problem

Terminal-bench was failing with all tasks showing 0 input/output tokens because the agent was exiting immediately after receiving the user message, without making any API calls.

Symptoms:

  • Latest nightly runs (Nov 5-7): All 80 tasks failed with agent_timeout
  • Agent ran for only ~45 seconds then exited
  • total_input_tokens: 0, total_output_tokens: 0
  • Stream started with caught-up and user message, but no stream-delta or stream-end events

Root cause:
The agentSessionCli.ts reads the user message from stdin via a pipe:

printf '%s' "$instruction" | bun src/debug/agentSessionCli.ts ...

Once stdin reaches EOF and is consumed, Bun detects no other active handles keeping the event loop alive and exits the process, even though async work (API streaming) is still pending.

Solution

Add an explicit keepalive interval that ensures the process stays alive until main() completes. The interval runs far into the future (1000 seconds) but gets cleared in the finally block once the agent session finishes.

Testing

Before fix:

  • Run #19173435224: 1 task, 0 tokens, ~2 min total (agent ran 45s)
  • Agent exited immediately after user message

After fix:

  • Run #19173548174: 1 task, resolved: true, ~7 min total (agent ran 3m17s)
  • 22 tool calls made
  • Stream-delta events present
  • Agent completed successfully

Related

Generated with cmux

Terminal-bench was failing with all tasks showing 0 input/output tokens
because Bun was exiting immediately after stdin closed, even though async
work (API calls) was still pending.

The agentSessionCli reads the user message from stdin, then waits for
stream completion. However, once stdin reaches EOF and is consumed, Bun
may exit if it detects no other active handles keeping the event loop alive.

Add an explicit keepalive interval that ensures the process stays alive
until main() completes. This interval runs far into the future but gets
cleared in the finally block once the agent session finishes.

This fixes the nightly terminal-bench failures where all 80 tasks were
failing with agent_timeout and 0 tokens.
@ammar-agent ammar-agent added this pull request to the merge queue Nov 7, 2025
github-merge-queue bot pushed a commit that referenced this pull request Nov 7, 2025
## Problem

Terminal-bench was failing with all tasks showing 0 input/output tokens
because the agent was exiting immediately after receiving the user
message, without making any API calls.

**Symptoms:**
- Latest nightly runs (Nov 5-7): All 80 tasks failed with
`agent_timeout`
- Agent ran for only ~45 seconds then exited
- `total_input_tokens: 0`, `total_output_tokens: 0`
- Stream started with `caught-up` and `user` message, but no
`stream-delta` or `stream-end` events

**Root cause:**
The `agentSessionCli.ts` reads the user message from stdin via a pipe:
```bash
printf '%s' "$instruction" | bun src/debug/agentSessionCli.ts ...
```

Once stdin reaches EOF and is consumed, Bun detects no other active
handles keeping the event loop alive and exits the process, **even
though async work (API streaming) is still pending**.

## Solution

Add an explicit keepalive interval that ensures the process stays alive
until `main()` completes. The interval runs far into the future (1000
seconds) but gets cleared in the finally block once the agent session
finishes.

## Testing

**Before fix:**
- Run #19173435224: 1 task, 0 tokens, ~2 min total (agent ran 45s)
- Agent exited immediately after user message

**After fix:**
- Run #19173548174: 1 task, **resolved: true**, ~7 min total (agent ran
3m17s)
- 22 tool calls made
- Stream-delta events present
- Agent completed successfully

## Related

- Fixes nightly terminal-bench failures from Nov 5-7
- Related to PR #507 (dist/ in archive) and PR #513 (build step in
workflow)

_Generated with `cmux`_
@ammario ammario removed this pull request from the merge queue due to a manual request Nov 7, 2025
@ammario ammario merged commit 4eb9c1c into main Nov 7, 2025
13 checks passed
@ammario ammario deleted the fix-agent-stdin-keepalive branch November 7, 2025 16:11
ibetitsmike pushed a commit that referenced this pull request Nov 7, 2025
## Problem

Terminal-bench was failing with all tasks showing 0 input/output tokens
because the agent was exiting immediately after receiving the user
message, without making any API calls.

**Symptoms:**
- Latest nightly runs (Nov 5-7): All 80 tasks failed with
`agent_timeout`
- Agent ran for only ~45 seconds then exited
- `total_input_tokens: 0`, `total_output_tokens: 0`
- Stream started with `caught-up` and `user` message, but no
`stream-delta` or `stream-end` events

**Root cause:**
The `agentSessionCli.ts` reads the user message from stdin via a pipe:
```bash
printf '%s' "$instruction" | bun src/debug/agentSessionCli.ts ...
```

Once stdin reaches EOF and is consumed, Bun detects no other active
handles keeping the event loop alive and exits the process, **even
though async work (API streaming) is still pending**.

## Solution

Add an explicit keepalive interval that ensures the process stays alive
until `main()` completes. The interval runs far into the future (1000
seconds) but gets cleared in the finally block once the agent session
finishes.

## Testing

**Before fix:**
- Run #19173435224: 1 task, 0 tokens, ~2 min total (agent ran 45s)
- Agent exited immediately after user message

**After fix:**
- Run #19173548174: 1 task, **resolved: true**, ~7 min total (agent ran
3m17s)
- 22 tool calls made
- Stream-delta events present
- Agent completed successfully

## Related

- Fixes nightly terminal-bench failures from Nov 5-7
- Related to PR #507 (dist/ in archive) and PR #513 (build step in
workflow)

_Generated with `cmux`_
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants