🤖 feat: stream advisor output live

[ThomasK33]

Summary

Streams nested advisor model response text into the advisor tool card while the tool is still running, without changing the final advisor tool result contract returned to the parent model.

Background

Advisor calls previously showed phase changes only and withheld the actual advice text until tool-call-end. This made long advisor consultations feel opaque even though the nested model was already generating useful text.

Implementation

Adds a UI-only advisor-output chat event, emits it from the advisor tool's streamText() chunks, forwards it through the workspace chat stream, stores it as bounded transient frontend state, and renders it in the expanded advisor tool card until the final persisted tool result arrives.

Validation

• env MUX_ESLINT_CONCURRENCY=2 make static-check
• bun test src/node/services/tools/advisor.test.ts
• bun test src/browser/stores/WorkspaceStore.test.ts
• bun test src/browser/features/Tools/AdvisorToolCall.test.tsx
• make typecheck
• targeted eslint over touched files
• make fmt-check
• git diff --check
• Dogfooded in an isolated dev-server sandbox with GPT-5.5 main model and GPT-5.5 advisor model at low reasoning; captured video and screenshots showing advisor text appearing while the tool remained in waiting for response.

Risks

Risk is concentrated in chat stream/transient UI state. The new event is UI-only, bounded, cleared on final advisor tool result and stale-message cleanup, and leaves model-visible tool output unchanged.

Pains

Default lint concurrency was OOM-killed in this workspace, so local lint/static-check was run with MUX_ESLINT_CONCURRENCY=2.

---

📋 Implementation Plan

Stream Advisor Responses to the Client

Goal

Stream the nested advisor model's response text to the client UI while the advisor tool is still running, without changing what the parent model receives from the tool.

Today the advisor tool is non-streaming from the user's perspective: the backend emits live phase changes (preparing_context, waiting_for_response, finalizing_result), then the UI receives the full advice only when the normal tool-call-end result arrives. The desired UX is for the advisor tool card to show advice text incrementally as the nested advisor model produces it.

Current implementation evidence

Verified implementation points from the repo:

• src/node/services/tools/advisor.ts
  • createAdvisorTool() currently calls generateText() for the nested advisor request.
  • It emits only advisor-phase UI events during execution.
  • It returns a final tool result shaped like { type: "advice", advice, advisorModel, reasoningLevel, remainingUses }.
  • It reports advisor model usage through config.reportModelUsage after the nested call completes.
• src/node/services/aiService.ts
  • Advisor eligibility is computed from experiment state, per-agent defaults, and advisorModelString.
  • advisorRuntime is injected into getToolsForModel() only when eligible.
  • Existing onAdvisorChunk captures the parent model's same-step text/reasoning before the advisor tool call; it does not stream advisor output.
• src/common/orpc/schemas/stream.ts
  • AdvisorPhaseEventSchema already defines a UI-only advisor event carried over workspace.onChat.
  • WorkspaceChatMessageSchema is the discriminated union that must include any new chat-stream event.
• src/node/services/agentSession.ts
  • AIService events are forwarded into workspace chat subscriptions; advisor-phase is already forwarded.
• src/browser/stores/WorkspaceStore.ts
  • Transient live state already exists for liveAdvisorPhase.
  • The store buffers live stream events until caught-up, updates live phase on advisor-phase, and clears advisor phase on tool-call-end for advisor.
• src/browser/features/Tools/AdvisorToolCall.tsx
  • The component renders phase/timer while executing and renders final Markdown advice only after the final result is available.

Advisor review status

An advisor review was requested after drafting this plan. Two advisor tool calls were attempted with compact review prompts, but both failed before returning advice with a sanitized provider error: Advisor request failed: Invalid JSON response with binary-like payload content. No advisor-originated recommendations were available to incorporate.

Because the review tool itself failed, the plan below is conservative and self-reviewed against the verified code paths. If the advisor tool becomes healthy before implementation starts, retry the review using the same summary and incorporate any concrete feedback before coding.

Recommended approach

Approach A — UI-only advisor-output event + nested streamText() (recommended)

Product LoC estimate: net +300 to +450 product LoC.

Implement a new UI-only chat event, tentatively named advisor-output, that carries advisor text deltas from the backend to the renderer while the tool is running. Replace the nested advisor generateText() call with streamText() and use onChunk (or an equivalent fullStream loop) to emit output chunks as they arrive. Accumulate those same chunks in the backend and preserve the existing final tool result shape for the parent model.

Why this is the best fit:

• Matches existing patterns (advisor-phase, bash-output, task-created) for UI-only tool progress.
• Does not conflate advisor output with normal assistant stream-delta events.
• Does not change the parent model's tool protocol: the parent still receives one final tool result after execute() resolves.
• Keeps persistence simple: final advice is still persisted through the normal dynamic-tool output part.
• Allows a small, surgical implementation with low risk to the main stream pipeline.

Alternatives considered

Approach B — Reuse tool-call-delta for advisor output

Product LoC estimate: net +180 to +280 product LoC, but not recommended.

tool-call-delta currently means streaming tool input args from the parent model, not tool output from a running tool. Reusing it would blur semantics and complicate existing stream processing/tests.

Approach C — Persist partial advisor output as mutable tool result state

Product LoC estimate: net +500 to +800 product LoC, not needed for the first version.

This would make reconnect/replay show in-progress advisor output exactly, but requires deeper changes to dynamic-tool persistence/partial writes and would risk coupling UI-only progress with model-visible tool output. Consider later only if live-only reconnect behavior is insufficient.

Scope

In scope:

• Stream advisor final text deltas to the current client session while advisor is running.
• Preserve the existing final advisor result contract.
• Preserve advisor usage reporting and per-turn cap behavior.
• Render live advisor text inside the existing AdvisorToolCall UI.
• Clear transient live output once the final tool-call-end result is available.
• Add tests for backend streaming, schema/store handling, and UI rendering.

Out of scope for this iteration:

• Streaming advisor reasoning/thinking to the UI.
• Streaming advisor chunks to the parent model as incremental tool output.
• Persisting every advisor chunk in history.
• Replaying missed advisor chunks after disconnect/reload before the final tool result exists.
• Changing advisor model selection, agent gating, or tool policy.

Detailed implementation plan

Phase 0 — Pre-flight decisions

Before coding, confirm these defaults unless the implementer discovers a strong reason to change them:

• Event name: advisor-output.
• Payload field for chunks: text rather than delta, matching bash-output's tool-output wording.
• Stream only advisor final text, not reasoning deltas.
• Do not persist partial advisor chunks; final tool-call-end remains the durable source of truth.
• Do not add a messageId in the first version; workspaceId + toolCallId is enough for transient UI state and matches existing advisor-phase scoping.

Phase 1 — Add a UI-only advisor output event contract

Files likely touched:

• src/common/orpc/schemas/stream.ts
• src/common/orpc/schemas.ts
• src/common/types/stream.ts
• src/common/orpc/types.ts

Steps:

1. Add a new schema near AdvisorPhaseEventSchema, for example:

   export const AdvisorOutputEventSchema = z.object({
     type: z.literal("advisor-output"),
     workspaceId: z.string(),
     toolCallId: z.string(),
     text: z.string(),
     timestamp: z.number(),
   });

2. Add AdvisorOutputEventSchema to WorkspaceChatMessageSchema.

3. Re-export/import the new schema through the aggregate schema module if needed (src/common/orpc/schemas.ts currently gathers stream schemas for downstream imports).

4. Export the inferred AdvisorOutputEvent type from src/common/types/stream.ts.

5. Add isAdvisorOutputEvent() to src/common/orpc/types.ts.

6. Keep the event UI-only: it should not be transformed into a MuxMessage and should not become model-visible history.

Defensive checks:

• Only emit non-empty text chunks.
• Keep workspaceId and toolCallId required so the renderer can scope chunks to a single tool card.
• Avoid adding messageId unless a later implementation needs replay dedupe; advisor-phase and bash-output already scope by workspace/toolCallId.

Quality gate:

• Run the type/schema tests that cover ORPC stream schemas, or at minimum make typecheck after Phase 2 when event producers exist.

Phase 2 — Stream inside the advisor backend tool

Files likely touched:

• src/node/services/tools/advisor.ts
• Tests in src/node/services/tools/advisor.test.ts

Steps:

1. Replace the nested generateText() call with streamText() from the AI SDK.
2. Keep all existing request inputs the same:
   • model
   • system: ADVISOR_SYSTEM_PROMPT
   • messages
   • tools: {}
   • providerOptions
   • parent/combined abort signal
   • optional maxOutputTokens
3. Use onChunk to capture text-delta chunks from the advisor stream.
   • Extract text defensively from text, delta, or textDelta because current stream code already handles provider/SDK variation this way.
   • Ignore empty chunks.
4. For every non-empty text delta:
   • append it to a local adviceText accumulator,
   • emit { type: "advisor-output", workspaceId, toolCallId, text: delta, timestamp: Date.now() } via config.emitChatEvent if available.
5. Await the stream's final text to guarantee completion.
   • Prefer const streamResult = streamText(...); const finalAdvice = await streamResult.text; while using onChunk for live updates.
   • Assert or reconcile that finalAdvice is non-empty or equal to the accumulated text when practical. Do not fail a successful call solely because chunk accumulation differs; use the final SDK text as the source of truth for the returned tool result.
6. Await usage/provider metadata after stream completion and feed them into the existing reportModelUsage path.
   • Use streamResult.usage or streamResult.totalUsage based on what best matches current generateText() semantics for a one-step, tool-less advisor call.
   • Keep provider metadata stamping unchanged in AIService.reportModelUsage.
7. Preserve existing error handling:
   • Abort returns Advisor request was aborted.
   • Other failures return sanitized Advisor request failed: ...
   • No usage is reported on failed advisor model calls.
8. Preserve existing phase events:
   • preparing_context before transcript/handoff work,
   • waiting_for_response immediately before starting streamText()/awaiting response,
   • finalizing_result after the nested stream finishes and before returning final result.

Defensive checks:

• Keep current assertions for advisorRuntime, non-empty model string, valid reasoning level, max uses, output token cap, transcript snapshot, and toolCallId.
• Add a narrow helper for chunk text extraction with tests, or reuse/extract an existing helper only if it keeps the change smaller than duplicating a few lines.
• Ensure concurrent advisor tool calls still respect usesThisTurn++ before awaits.

Quality gate:

• Update src/node/services/tools/advisor.test.ts to mock streamText() instead of generateText().
• Add/adjust tests for:
  • successful final result still has the same shape,
  • usage is reported once after completion,
  • advisor-output events are emitted for text chunks before final result,
  • empty chunks are ignored,
  • no output events are emitted if emitChatEvent, workspaceId, or toolCallId is unavailable,
  • abort/error behavior remains unchanged,
  • max output tokens are still passed.

Phase 3 — Forward advisor output events through workspace chat

Files likely touched:

• src/node/services/agentSession.ts
• Any typed AIService event declarations if present near event emission/listening code

Steps:

1. Add a forwarding branch mirroring advisor-phase:

   forward("advisor-output", (payload) => {
     this.markActiveStreamHadAnyOutput();
     this.emitChatEvent(payload);
   });

   If markActiveStreamHadAnyOutput() is too broad for UI-only advisor chunks, omit it; however bash-output already marks output, and advisor text is user-visible output from a running tool, so marking is reasonable.

2. Confirm the event traverses:

   • advisor tool config.emitChatEvent,
   • AIService.emit(event.type, event),
   • AgentSession forwarding,
   • workspace.onChat ORPC subscription.

Quality gate:

• Add/extend tests if there is an existing AgentSession forwarding coverage pattern. Otherwise rely on WorkspaceStore integration tests plus typecheck, since this forwarding path is identical to existing UI-only events.

Phase 4 — Store live advisor output in renderer transient state

Files likely touched:

• src/browser/stores/WorkspaceStore.ts
• src/browser/stores/WorkspaceStore.test.ts

Steps:

1. Add a transient output state type, for example:

   export interface AdvisorLiveOutputState {
     text: string;
     timestamp: number;
   }

2. Add liveAdvisorOutput: Map<string, AdvisorLiveOutputState> to WorkspaceChatTransientState.

3. Initialize it in the transient state factory.

4. Add getAdvisorToolLiveOutput(workspaceId, toolCallId) and useAdvisorToolLiveOutput(...).

   • Return the stored object reference, not a freshly allocated object, to avoid useSyncExternalStore loops.

5. Treat advisor-output as a buffered event before caught-up, like bash-output, advisor-phase, and task-created.

6. In processStreamEvent():

   • identify isAdvisorOutputEvent(data),
   • append data.text to the previous live output for that toolCallId,
   • update timestamp,
   • schedule a throttled/idle state bump rather than immediate re-render on every token.

7. Clear liveAdvisorOutput:

   • on tool-call-end for advisor, alongside liveAdvisorPhase,
   • on full replay / no active stream / stream mismatch where liveAdvisorPhase is already cleared.

Defensive checks:

• Ignore zero-length text defensively in the store even if backend filters it.
• Preserve stable snapshots for unchanged events.
• Do not append chunks after the final tool result is processed for the same tool call.

Quality gate:

• Extend WorkspaceStore.test.ts with tests for:
  • accumulating multiple advisor-output chunks,
  • buffering before caught-up then applying in order,
  • clearing on advisor tool-call-end,
  • clearing on full replay/no active stream cleanup.

Phase 5 — Render live advisor text in the tool UI

Files likely touched:

• src/browser/features/Tools/AdvisorToolCall.tsx
• src/browser/features/Tools/AdvisorToolCall.test.tsx

Steps:

1. Import/use useAdvisorToolLiveOutput(workspaceId, toolCallId).
2. Compute a liveAdviceText only while the tool is executing and final advisorResult is not present.
3. Render live text in the existing expanded details area:
   • show the same Advice section when liveAdviceText is non-empty,
   • render through MarkdownRenderer with preserveLineBreaks, matching final advice,
   • optionally show a small LoadingDots indicator next to the section label or below the text.
4. Keep final result behavior unchanged:
   • once advisorResult?.type === "advice", render advisorResult.advice as today,
   • final result should supersede and clear live text via store cleanup.
5. Use live text as ToolDetails text fallback so copy/selection/preview behavior works while streaming, if that prop is intended for that purpose.
6. Keep collapsed header focused on phase/timer. Do not stream full text into the collapsed header.

Performance guardrail:

• Because Markdown rendering can be expensive, rely on WorkspaceStore throttling for high-frequency output. If dogfooding shows jank, switch live rendering to a plain text/pre-wrap block and keep Markdown rendering for the final result only.

Quality gate:

• Extend AdvisorToolCall.test.tsx to verify:
  • live output renders while status="executing" and final result is absent,
  • final advice still renders after completion,
  • final advice supersedes live output,
  • the fallback "Running"/phase behavior still works when no live output exists.

Phase 6 — Validation and dogfooding

Automated validation

Run targeted tests first, then broader validation:

1. bun test src/node/services/tools/advisor.test.ts
2. bun test src/browser/features/Tools/AdvisorToolCall.test.tsx
3. bun test src/browser/stores/WorkspaceStore.test.ts
4. make typecheck
5. make lint
6. If the diff is small and time allows, make test or make static-check.

Fix failures before claiming success.

Manual dogfooding setup

1. Start a development Mux instance in an isolated workspace.
   • Preferred: use the repo's desktop/dev-server sandbox skills if available to avoid interfering with the user's running Mux.
   • Fallback: run the normal dev target from this repo, e.g. make dev, following current project instructions.
2. Enable the advisor-tool experiment in Settings → Experiments.
3. Configure an advisor model in the advisor settings UI.
4. Ensure the active agent has Advisor enabled in Settings → Tasks.
5. Open a test workspace and ask a prompt that should naturally call advisor, for example:
   • "Plan a risky refactor and use advisor before deciding."
6. Use agent-browser/Electron automation to capture:
   • an initial screenshot/video frame showing the advisor tool card in a running phase,
   • a screenshot/video while advisor text is partially streamed in the expanded card,
   • a final screenshot after tool-call-end showing the final advice result.
7. Attach screenshots and the recording artifact for reviewer verification.

Real dogfooding continuation request (Plan Mode note)

The user requested a real dogfooding session after implementation, specifically:

• read the dogfood, agent-browser, and dev-server-sandbox skills;
• use agent-browser skills get core before browser automation;
• start a real isolated Mux dev-server sandbox rather than a synthetic component-only page;
• configure providers using the keys already present, preferably selecting a faster advisor model so streaming chunks are visible quickly;
• record a video of the real advisor tool streaming output and attach that video in chat.

Plan Mode constraints currently prevent executing this request directly: only the plan file may be edited, and bash must remain read-only. A real dogfood session requires non-read-only operations such as starting a sandbox that creates a temporary MUX_ROOT, copying provider/project config, changing sandbox settings/provider choices, creating dogfood output directories/artifacts, and recording video files. The next step therefore requires switching back to Exec mode.

Dogfood-only approach — real sandbox session

Product LoC estimate: net +0 product LoC, unless dogfooding uncovers a bug that must be fixed.

Execution plan once in Exec mode:

1. Re-check working tree and preserve all intentional formatter changes noted in the continuation message.
2. Start an isolated Mux instance with make dev-server-sandbox.
   • Use the default sandbox seeding behavior so provider config from the developer root is copied when present.
   • Capture the printed MUX_ROOT, backend port, and Vite URL without exposing secrets.
3. Open the sandbox URL with a named agent-browser session.
   • Start with agent-browser skills get core guidance: open, snapshot -i, interact by refs, re-snapshot after changes.
4. Configure runtime/settings in the UI:
   • enable the advisor tool experiment if not already enabled;
   • configure/select a provider/model using only keys already present in the sandboxed provider config;
   • choose a fast advisor model when available (for example a fast OpenAI/Anthropic small model) so partial chunks appear quickly;
   • ensure the active agent permits advisor usage.
5. Create or select a disposable workspace in the sandbox UI.
6. Start a video recording before the user-visible advisor interaction:
   • output path under a dogfood artifacts directory outside the repo or in an ignored temp location;
   • record the whole flow from prompt submit through live advisor streaming and final result.
7. Send a prompt that strongly encourages advisor use, e.g. “Plan a risky refactor and use advisor before deciding; keep the advisor answer long enough that streaming is visible.”
8. During recording, expand the advisor tool card and verify:
   • phase/status appears first;
   • text chunks appear before final tool-call-end;
   • final completed advice supersedes the live text cleanly;
   • parent assistant continues normally after the advisor result.
9. Capture supporting screenshots at the key states (initial running, partial streamed advice, final result) and check browser console/errors.
10. Stop the video recording, attach the video in chat, and optionally attach the best screenshot.
11. If dogfooding reveals defects, return to implementation with a focused TDD fix, then repeat targeted validation and the real dogfood capture.

Dogfood quality gates:

• Real sandbox, not synthetic component page.
• Provider/model settings configured from existing keys only; no keys printed in logs or chat.
• Video artifact attached in chat.
• Screenshots/video show actual advisor tool card streaming text incrementally.
• Console/errors checked and summarized.
• No repo-untracked dogfood artifacts left behind unless intentionally ignored/temp.

Dogfooding acceptance checks

• Advisor text appears incrementally before the final tool result.
• The final tool result exactly matches the completed advice shown live.
• The main assistant output does not show advisor text as normal assistant stream-delta content.
• The parent assistant continues after the advisor tool completes and can use the final advice.
• Interrupting the stream while advisor is running returns an advisor abort/error result and does not leave stale live output in the next turn.
• Reconnecting/reloading during advisor execution does not crash; at worst, in-progress live chunks are missing until final result arrives.

Acceptance criteria

• The advisor backend uses streaming for nested advisor model responses.
• UI receives and renders advisor text chunks before the final tool-call-end.
• Final advisor tool result shape remains backward-compatible.
• Parent model behavior is unchanged: it receives only the completed advisor result after tool execution.
• No advisor chunks are persisted as normal assistant text or sent to future model calls outside the final tool result.
• Usage/cost reporting for advisor model calls remains correct and isolated under the advisor model/tool bucket.
• Existing advisor phase display still works.
• Live advisor output is transient and cleared when the final advisor result arrives.
• Automated tests cover backend streaming, renderer state accumulation/cleanup, and UI rendering.
• Dogfooding evidence includes screenshots and a video/recording of live advisor output streaming.

Risks and mitigations

Risk	Mitigation
AI SDK streamText() result usage/provider metadata differs from generateText()	Add focused unit tests around usage reporting; use a one-step tool-less stream to match current semantics.
Markdown rendering every chunk causes UI jank	Throttle store bumps; if needed, render live output as plain text and final output as Markdown.
Event ordering before caught-up causes missing chunks	Add advisor-output to buffered event handling before caught-up.
Reconnect during advisor execution loses transient chunks	Accept for first version; final tool result remains authoritative. Document as non-goal.
Accidentally exposing advisor text as normal assistant text	Use a distinct UI-only event, never stream-delta. Add store/UI tests.
Parent model expects final tool result and cannot consume partial output	Preserve execute() return shape and treat streaming as UI-only.
Duplicated final/live text	Clear transient output on tool-call-end; final result supersedes live output.

Suggested implementation order

1. Event schema/type guard (advisor-output).
2. Backend advisor streaming + unit tests.
3. AgentSession forwarding.
4. WorkspaceStore transient state + tests.
5. AdvisorToolCall rendering + tests.
6. Typecheck/lint/targeted tests.
7. Retry advisor review if the advisor tool has recovered; otherwise proceed with the documented failed-review caveat.
8. Manual dogfooding with screenshots/video.

This order keeps the backend producer and event contract testable before touching UI rendering, then validates the renderer pipeline with transient-state tests before dogfooding the full UX.

---

Generated with mux • Model: openai:gpt-5.5 • Thinking: xhigh • Cost: $139.03

[ThomasK33]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 136276da0c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Please take another look.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9a859ff16e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Please take another look.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1d3bc040ca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

Please take another look.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Swish!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".