Skip to content

v0.5.0 — Streaming Thinking UI & Post-Tool Speed Optimization

Choose a tag to compare

@pluginslab pluginslab released this 09 Mar 12:01
· 362 commits to main since this release

What's New

Streaming Thinking Blocks

  • LLM reasoning (<think> blocks) now streams live in the chat UI — shown expanded with a purple pulsing timeline dot while generating, then collapses into a peekable "Thought process" entry
  • Same collapsible pattern as tool call results — click to expand/peek at reasoning
  • Thinking persisted as THINKING message type in the session timeline
  • Works in both ReAct (tool selection) and conversational (direct LLM) paths

Post-Tool /nothink Optimization

  • After tool results are available, thinking is automatically disabled on subsequent LLM calls
  • The LLM has concrete data and only needs to summarize — reasoning is unnecessary overhead
  • Saves ~8-10s per interaction on the final answer generation
  • Empty <think> blocks suppressed from the UI when thinking is disabled

3-Tier Message Routing

  • Workflow detection → ReAct (with/without thinking) → conversational (direct LLM)
  • Clear action commands skip thinking for speed; ambiguous inputs keep thinking for accuracy
  • Knowledge questions bypass ReAct entirely

Other Improvements

  • ReAct agent switched to streaming LLM calls for live thinking display
  • Per-message speed stats (prefill/decode tokens per second)
  • Verbose loading descriptions removed, cache detection fixed

Technical Details

  • New THINKING message type in ChatSession
  • disableThinkingAfterTool: true in REACT_CONFIG (enabled by default)
  • onThinkingStart/Chunk/End callback chain through ReactAgent → ChatOrchestrator → ChatContainer
  • Purple pulsing dot (--thinking) with agentic-pulse CSS animation