feat: Enhance summarization and asking features with improved fallbac…

[rostilos]

…k handling and retry logic

Summary by CodeRabbit

• Bug Fixes
  • Improved robustness of command execution when handling streaming outputs and structured responses
  • Added intelligent fallback mechanisms to gracefully handle empty or invalid results with automatic retry logic
  • Enhanced error recovery to ensure consistent and reliable command processing across edge cases and failure scenarios

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7f2b90c6-2a96-4012-95fe-67990775415d

📥 Commits

Reviewing files that changed from the base of the PR and between b85b5b1 and 7ea5a2c.

📒 Files selected for processing (2)

• python-ecosystem/inference-orchestrator/src/service/command/command_service.py
• python-ecosystem/inference-orchestrator/tests/test_command_service.py

---

📝 Walkthrough

Walkthrough

This PR enhances command execution resilience by refactoring streaming output handling for both summarize and ask flows. New text-extraction utilities normalize MCP/LangChain stream item formats, updated execute methods coerce results to required schemas, and explicit retry/fallback logic handles empty or invalid structured responses by retrying without schema or invoking the LLM directly.

Changes

Command Execution Resilience

Layer / File(s)	Summary
Text content extraction utilities src/service/command/command_service.py	_extract_agent_item_text and _coerce_text_content extract plain text from diverse stream item representations including dicts with common keys, messages[-1] arrays, content/model_dump() objects, and block lists.
Summarize execution resilience src/service/command/command_service.py, tests/test_command_service.py	_execute_summarize now extracts text from streaming items and normalizes via _coerce_summarize_final_result/_summary_or_empty_error into {summary, diagram, diagramType}. When normalization fails, it retries via agent.run without output_schema or falls back to direct llm.ainvoke. Tests verify dict-based stream extraction, retry on empty summaries/exceptions, and LLM fallback behavior.
Ask execution resilience src/service/command/command_service.py, tests/test_command_service.py	_execute_ask now extracts text from streaming items and normalizes via _coerce_ask_final_result/_answer_or_empty_error into {answer}. When normalization fails, it retries via agent.run without output_schema or falls back to direct llm.ainvoke. Tests verify dict-based stream extraction, retry on empty answers, and LLM fallback behavior.

Sequence Diagram

sequenceDiagram
  participant Agent
  participant TextExtraction
  participant Coercion
  participant LLM
  Agent->>TextExtraction: stream items
  TextExtraction->>Coercion: extracted text
  Coercion->>Coercion: normalize to schema
  alt normalization succeeds
    Coercion-->>Agent: return result
  else normalization empty/invalid
    Agent->>Agent: retry via agent.run (no schema)
    alt retry succeeds
      Agent-->>Coercion: return result
    else retry empty
      Agent->>LLM: ainvoke fallback prompt
      LLM->>Coercion: coerce LLM response
      Coercion-->>Agent: return result
    end
  end

Loading

---

🎯 3 (Moderate) | ⏱️ ~20 minutes

---

🐰 A hop through the streams so bright,
Text extracted, schemas coerced right,
When outputs are bare, retries abound,
LLM fallbacks—resilience found! ✨

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 1.5.7-rc

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}