Skip to content

Fix: restore token-by-token streaming for chains using createTextOnlyOutputParser#6086

Open
HenryHengZJ wants to merge 1 commit intomainfrom
bugfix/Output-Parser-Streaming
Open

Fix: restore token-by-token streaming for chains using createTextOnlyOutputParser#6086
HenryHengZJ wants to merge 1 commit intomainfrom
bugfix/Output-Parser-Streaming

Conversation

@HenryHengZJ
Copy link
Copy Markdown
Contributor

  • createTextOnlyOutputParser() used a RunnableLambda which accumulates all LLM tokens before yielding, breaking progressive streaming in any chain that relies on streamLog() (e.g. Conversational Retrieval QA Chain)
  • Replaced with a proper Runnable subclass (TextOnlyOutputParser) that implements _transform() to yield each token as it arrives
  • Reasoning-model filtering (stripping thinking/reasoning blocks) is preserved

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the createTextOnlyOutputParser utility to improve streaming support by replacing the RunnableLambda implementation with a dedicated TextOnlyOutputParser class. The logic for extracting text from message chunks has been moved to a helper function. A review comment suggests enhancing type safety by replacing the any type with a type guard during the filtering of content blocks.

@HenryHengZJ
Copy link
Copy Markdown
Contributor Author

This is a fix coming from here

@HenryHengZJ HenryHengZJ requested a review from yau-wd March 30, 2026 16:45
@estebanjosse
Copy link
Copy Markdown
Contributor

I tested this PR locally and can confirm it resolves #6070 on my side.

Streaming is now working again as expected in the Conversational Retrieval QA Chain (token-by-token output in the UI).

For context, I initially investigated this issue and opened an alternative fix in #6089, but I only saw this PR afterwards.

This approach also resolves the issue correctly 👍

I also added a test harness covering this behavior in my PR — I’d be happy to contribute/adapt those tests to this PR if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression: Streaming broken in Conversational Retrieval QA Chain after upgrade from 3.0.13 to 3.1.x

2 participants