Skip to content

Fix streaming output when usestream=true (support plain text deltas)#116

Merged
nmaguiar merged 3 commits intomainfrom
codex/assess-chatbotmode-and-usestream-interactions
Mar 16, 2026
Merged

Fix streaming output when usestream=true (support plain text deltas)#116
nmaguiar merged 3 commits intomainfrom
codex/assess-chatbotmode-and-usestream-interactions

Conversation

@nmaguiar
Copy link
Copy Markdown
Contributor

@nmaguiar nmaguiar commented Mar 15, 2026

Motivation

  • The JSON-only stream parser expected an "answer" field and could suppress visible output when the provider emitted plain text deltas (e.g., via promptStreamWithStats), causing usestream=true sessions (notably chatbotmode=true) to show no streamed content.
  • The web UI expects SSE stream events to be emitted incrementally, so backend suppression produced an answered-but-not-displayed symptom in the frontend.

Description

  • Added a plain passthrough streaming handler MiniA.prototype._createPlainStreamDeltaHandler that emits raw text deltas immediately and inserts a leading newline on first output.
  • Updated the agent streaming selection to choose between the JSON-aware handler (_createStreamDeltaHandler) and the plain handler depending on whether promptStreamJSONWithStats (JSON stream) or promptStreamWithStats (plain stream) is being used and the noJsonPrompt flags.
  • Applied the same handler-selection logic in both the core agent loop (_startInternal) and the chatbot loop (_runChatbotMode) so chatbotmode=true benefits from the fix.
  • Preserved existing JSON/markdown-aware behavior for JSON envelope streams while ensuring plain token streams are displayed incrementally.

Testing

  • Ran node --check mini-a.js to verify the modified file passes a Node syntax check and it succeeded.
  • Attempted to run the project utility test job ojob tests/miniAUtils.yaml, but the ojob command is not available in this environment so the test job could not be executed here.

- Add guard to detect and skip re-registering proxy-dispatch on LLM instances
- Mark LLMs as registered and preserve a bare _llmNoTools snapshot to avoid duplicate attachments
- Add fallbacks for extracting `answer` and `params` from `arguments` and adjust streaming/tool invocation logic for tool-calling compatibility

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@nmaguiar nmaguiar marked this pull request as ready for review March 16, 2026 02:17
…ions

  - Add _extractToolCallActions() to recursively extract tool calls from
    nested payloads with deduplication and allowed-tools filtering
  - Add _resolveDebugChannelName() and _snapshotDebugChannel() for debug
    channel handling
  - Improve _runChatbotMode() to extract tool actions from responses when
    no explicit actions are present
  - Add fallback logic when model requests tool calling but no execution
    completes, switching to action-based mode
  - Track extracted response text and tool call detection for better
    interaction flow
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes missing incremental streamed output when usestream=true by handling providers that emit plain text deltas (non-JSON), ensuring SSE stream events are emitted as text arrives.

Changes:

  • Added a plain passthrough streaming delta handler and updated streaming handler selection to support non-JSON delta streams.
  • Adjusted streaming/prompt selection logic in both agent mode (_startInternal) and chatbot mode (_runChatbotMode).
  • Introduced additional tool-calling/MCP-related robustness changes (tool-call extraction helpers, MCP proxy registration de-dupe, and recovery fallbacks).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread mini-a.js
Comment thread mini-a.js
Comment thread mini-a.js
Comment thread mini-a.js
Comment thread mini-a.js
Comment on lines +12110 to +12114
// When function calling is active (_useToolsActual), skip format:json constraint —
// sending both "tools" and "format:json" simultaneously breaks Ollama thinking models
// (e.g. qwen3.x) which return {"error":"..."} when these modes conflict.
var skipJsonFormat = this._useToolsActual === true
if (canStreamJson && !noJsonPromptFlag && !skipJsonFormat) {
@nmaguiar nmaguiar force-pushed the codex/assess-chatbotmode-and-usestream-interactions branch from 200418b to 77c875e Compare March 16, 2026 04:25
@nmaguiar nmaguiar merged commit bf4a09f into main Mar 16, 2026
4 checks passed
@github-project-automation github-project-automation bot moved this from Backlog to Done in Continuous Enhancement Mar 16, 2026
@nmaguiar nmaguiar deleted the codex/assess-chatbotmode-and-usestream-interactions branch March 16, 2026 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Development

Successfully merging this pull request may close these issues.

2 participants