Skip to content

fix(broker): stop worker stderr from rendering inside agent xterm#931

Merged
khaliqgant merged 1 commit into
mainfrom
fix/worker-stderr-leaks-into-pty-output
May 20, 2026
Merged

fix(broker): stop worker stderr from rendering inside agent xterm#931
khaliqgant merged 1 commit into
mainfrom
fix/worker-stderr-leaks-into-pty-output

Conversation

@khaliqgant
Copy link
Copy Markdown
Member

Summary

  • The pty_worker subprocess is launched with stderr: Stdio::piped() (crates/broker/src/worker.rs:380-382) and read by spawn_worker_reader(..., parse_json=false, ...). The fallback branch (worker.rs:961) was wrapping every non-JSON line as {"type":"worker_stream","payload":{"stream":"stderr","chunk": line}} and forwarding it to dashboards.
  • Dashboards treat any worker_stream chunk as PTY output and append it to the agent's xterm buffer. The result: tracing logs from the pty_worker (e.g. the idle watchdog "watchdog: no PTY output for Ns — marking idle") render inside the agent terminal, on top of the CLI's input prompt.
  • Fix the leak at the source (stderr is for diagnostics, not PTY output) and quiet the watchdog so it doesn't even hit the log file under the default RUST_LOG=info filter.

Changes

  • crates/broker/src/worker.rs — in spawn_worker_reader, skip the worker_stream fallback when parse_json is false. The line is still written to the per-worker log file, so debugging context is preserved.
  • crates/broker/src/pty_worker.rs — downgrade the two idle-watchdog tracing lines from info/warn to debug. The send_frame("agent_idle" | "agent_blocked_on_send", …) calls are unchanged, so dashboards still receive the state transitions.

Test plan

  • cargo build -p agent-relay-broker
  • Spawn an agent via a dashboard (e.g. Pear), leave it idle past NO_OUTPUT_EXIT_TIMEOUT, and confirm no watchdog: … text appears in the xterm buffer.
  • Confirm agent_idle / agent_blocked_on_send events still arrive in the dashboard (state UI still updates).
  • cargo test -p agent-relay-broker (no behavioural change expected for existing tests).

The pty_worker subprocess is spawned with `stderr: Stdio::piped()` and
read by `spawn_worker_reader(..., parse_json=false, ...)`. Any non-JSON
line from that reader (i.e. every stderr line, including
`tracing::info!`/`warn!` output) was wrapped as a `worker_stream` event
and forwarded to dashboards, which append `worker_stream` chunks to the
agent's xterm buffer. Tracing logs like the idle watchdog therefore
rendered on top of the CLI's input prompt, breaking the agent UI.

Two changes:

- `spawn_worker_reader`: when `parse_json` is false (the stderr reader),
  keep writing the line to the worker log file but skip the
  `worker_stream` fallback. Stderr is for diagnostics, not PTY output.
- `pty_worker` idle watchdog: downgrade the two
  `"watchdog: no PTY output …"` lines from `info`/`warn` to `debug` so
  they stay out of the log file under the default `RUST_LOG=info`
  filter. The `send_frame("agent_idle" | "agent_blocked_on_send", …)`
  calls are unchanged, so dashboards still receive the state events.
@khaliqgant khaliqgant requested a review from willwashburn as a code owner May 20, 2026 19:21
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 20, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 75193a07-558f-4da1-918b-bd19961a9aaa

📥 Commits

Reviewing files that changed from the base of the PR and between 8cc03ca and 7a2dece.

📒 Files selected for processing (2)
  • crates/broker/src/pty_worker.rs
  • crates/broker/src/worker.rs

📝 Walkthrough

Walkthrough

This PR reduces logging noise and filters stream event emission in worker output handling. PTY watchdog timeout conditions downgrade from warn/info to debug severity, and stderr processing is modified to prevent non-stdout content from being forwarded as worker_stream events, preserving only PTY-protocol output for client streams.

Changes

Worker Output Handling

Layer / File(s) Summary
PTY watchdog logging downgrade
crates/broker/src/pty_worker.rs
Watchdog timeout log levels reduced from warn! to debug! (blocked_on_send case) and from info! to debug! (idle timeout case), reducing operational noise while retaining diagnostics.
Stderr stream filtering
crates/broker/src/worker.rs
spawn_worker_reader adds early-return guard for non-JSON stderr mode to prevent stderr lines from being emitted as worker_stream events, ensuring only PTY output is forwarded to client streams.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Suggested reviewers

  • willwashburn

Poem

🐰 Quiet now, the logs declare,
No warning screams fill the air—
Stderr stays, but won't betray
The stream, which flows a cleaner way.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: fixing worker stderr from rendering inside the agent xterm terminal.
Description check ✅ Passed The description is comprehensive, covering summary, changes, and test plan sections that align well with the template requirements.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/worker-stderr-leaks-into-pty-output

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

@khaliqgant khaliqgant merged commit a8bd343 into main May 20, 2026
40 checks passed
@khaliqgant khaliqgant deleted the fix/worker-stderr-leaks-into-pty-output branch May 20, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant