Skip to content

[Story]: Show AI reasoning / thinking process during chat generation #248

Description

@adoLime

User Story

As a user chatting with the AI, I want to see the AI's reasoning ("thought process") while it generates an answer, so that I can follow how the answer is derived, build trust in the result, and better understand its sources and citations.

Context & Motivation

  • Type: Functional (AI transparency / RAG chat UX)
  • Parent epic: 04 EPIC - Knowledge Layer, RAG & Source Citation #53 (04 EPIC – Knowledge Layer, RAG & Source Citation)
  • Teams: team:ai (capture reasoning + emit SSE event), team:backend (passthrough DTO), team:frontend (render reasoning panel)
  • Current state: Chat streams only final-answer token + citation SSE events. While the AI "thinks", the user sees only a 3-dot indicator (brainrot fallback after 4s). The configured LLM providers can already emit a reasoning channel (Anthropic extended thinking, OpenAI-compatible reasoning_content via DeepSeek-R1/Qwen, Ollama thinking), but the AI service reads only the text channel and discards it (sprintstart-ai src/llm/openai_client.py:213, anthropic_client.py:187, ollama_client.py:252).
  • Scope decision: Live-only — reasoning is shown during generation and NOT persisted with the message (privacy + storage). Does not satisfy 02 EPIC - Security, Privacy & Local AI Governance #51's "stored traceable reasoning" AC (separate story later).
  • Related: [Story]: Settings hub  #242 (chat loading/typing indicator + cancel) — this story refines the "what the user sees while generating" portion with actual reasoning content. [Story]: Source filter in the chat #104 (source filters) is the sibling chat story under the same epic.

Acceptance Criteria

  • A new SSE event type (e.g. reasoning) is emitted by the AI service during generation, kept separate from token so reasoning and answer text are never mixed.
  • Reasoning is streamed live to the frontend and rendered in a collapsible "Thinking…" panel above/within the assistant bubble while the answer is generated.
  • While reasoning events arrive, they replace/shadow the 3-dot typing indicator; if the active model emits no reasoning, the chat behaves exactly as today (graceful no-op).
  • Reasoning text is NOT appended to the persisted assistant message content — only the final answer + citations are stored.
  • Backend passes the new event through unchanged (DTO field + when-branch) and does not persist reasoning.
  • Given a reasoning-capable model is configured, when the user sends a prompt, then reasoning chunks appear in real time before/alongside the answer.
  • Edge case: a long reasoning phase (>4s) keeps showing streamed reasoning instead of the brainrot fallback.
  • Edge case: malformed/empty reasoning deltas are skipped without breaking the stream.

Sub-Tasks (by team)

  • AI — extend the LLM stream() contract (src/llm/base.py:43) to surface reasoning separately; read delta.reasoning_content (openai), enable thinking=... + capture thinking-block deltas (anthropic), read chunk.message.thinking (ollama); emit {"type":"reasoning","content":...} via sse_event() in src/api/routes/chat.py:111 and src/agents/orchestrator.py:101.
  • Backend — add a reasoning field to AiStreamMessage (chat/models/responses/AiStreamMessage.kt:39) + passthrough branch in ChatAiClient.kt:80 and ChatService.kt:213 (must NOT append to the persisted content buffer at :215).
  • Frontend — add reasoning to the ChatEvent union (src/services/chatService.ts:39) + case (:116); add reasoning to ChatMessage and onReasoning to StreamHandlers (src/features/chatbot/types.ts); handler in useChat.ts:184; render a collapsible panel in ChatPage.tsx:175 (replace/supplement the indicator at :255).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requeststoryteam:aiAI / Retrieval team (sprintstart-ai, Python)team:backendBackend team (sprintstart-backend, Kotlin/Spring Boot)team:frontendFrontend team (sprintstart-frontend, React)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions