Fix the agent runner bug

[tysonthomas9]

Summary by CodeRabbit

Release Notes

• New Features
  • Agent summaries now provide comprehensive, multi-section analysis with detailed breakdowns of decisions, actions, and recommendations.
  • Action agent automatically includes structured summaries in responses for improved transparency.
  • Agent tool results now consistently display in the agent conversation lane, enhancing message organization and readability.

[claude]

Claude finished @tysonthomas9's task —— View job

---

Code Review Complete ✅

Overview

This PR addresses agent runner configuration bugs with three focused improvements to enhance agent execution feedback and UI behavior. The changes improve summary generation quality, enable detailed execution insights for action agents, and fix UI lane assignment consistency.

Detailed Analysis

1. AgentRunner.ts - Enhanced Summary Generation (+27/-12 lines)

Changes: Significantly expanded the summary prompt for max iteration scenarios from a basic summary to a comprehensive 400-600 word detailed analysis.

✅ Strengths:

• Much improved user experience: Users will now get actionable, detailed feedback instead of terse summaries when agents hit iteration limits
• Structured format: The new 5-section format (User Request, Agent Decisions, Final State, Progress Assessment, Recommendations) provides systematic analysis
• Specific guidance: Includes concrete instructions for minimum word count and detailed analysis requirements
• Actionable insights: The "Recommendations for Next Steps" section gives users clear guidance on how to proceed

⚠️ Considerations:

• Token cost impact: 400-600 word summaries will significantly increase token usage for summary generation calls. Consider if this cost is justified by the improved UX
• Response time: Longer summaries may increase generation time, potentially affecting user perception of responsiveness
• Consistency: Only applies to max_iterations case - consider if other completion types (final_answer, error) would benefit from similar detail levels

2. ActionAgent.ts - Enable Summary Generation (+1 line)

Changes: Added includeSummaryInAnswer: true to action agent configuration.

✅ Strengths:

• Execution insights: Users will get detailed analysis of what the action agent accomplished
• Debugging aid: Helpful for understanding why actions succeeded or failed
• Consistent with enhanced feedback theme: Aligns with the overall PR goal of better user feedback

⚠️ Considerations:

• Performance overhead: Summary generation adds an additional LLM call per action execution
• Token costs: Each action execution now triggers summary generation, increasing operational costs
• Action agent specificity: Consider if this level of detail is needed for simple actions like "click button"

3. AgentNodes.ts - UI Lane Assignment Fix (+1/-2 lines)

Changes: Simplified UI lane assignment logic to always route agent tool results to 'agent' lane, removing error-based routing exception.

✅ Strengths:

• UI consistency: Agent tool results always appear in the agent lane regardless of success/failure
• Simpler logic: Removes complex conditional that could cause confusion in the UI
• Better user experience: Users get consistent visual organization of agent vs. regular tool outputs

**✅ No concerns: This is a straightforward UI improvement with clear benefits

Code Quality Assessment

Excellent

• Clear intent: All changes directly address user feedback and debugging needs
• Minimal surface area: Changes are surgical and focused
• Good documentation: The enhanced prompt includes clear instructions and examples
• Type safety: No type-related issues introduced

Performance Considerations

• Token usage increase: The combination of detailed summaries + action agent summaries will significantly increase LLM API costs
• Latency impact: Additional summary generation calls may affect response times
• Recommendation: Consider making summary detail levels configurable or conditional based on user preferences/agent complexity

Security Assessment

• ✅ No security concerns: Changes are limited to prompt improvements and configuration flags
• ✅ No data exposure risks: Summary generation follows existing patterns

Testing Implications

• Unit tests: May need updates for action agent configuration changes
• Integration tests: Should verify UI lane assignment behavior works as expected
• Cost monitoring: Recommend adding metrics to track the token usage impact in production

Final Recommendation: APPROVE ✅

This is a well-crafted PR that directly improves user experience with agent execution feedback. The changes are focused, safe, and address real usability issues. While there are token cost implications, the enhanced debugging and user feedback capabilities justify the investment.

Suggested follow-ups:

1. Monitor token usage after deployment to quantify cost impact
2. Consider adding configuration options for summary detail levels
3. Evaluate extending detailed summary formatting to other completion types

---

[coderabbitai]

Walkthrough

This PR enhances agent feedback and output systems across three key files: expanding agent progress summarization from brief summaries to detailed 400–600 word multi-section analyses in AgentRunner.ts, enabling summary inclusion in ActionAgent responses, and modifying UI lane assignment for agent tool results to consistently display in the agent lane regardless of errors.

Changes

Cohort / File(s)	Summary
Agent summarization and output front_end/panels/ai_chat/agent_framework/AgentRunner.ts, front_end/panels/ai_chat/agent_framework/implementation/agents/ActionAgent.ts	Replaced concise max-iterations summary prompts with comprehensive 400–600 word multi-section analysis prompts (User Request, Agent Decisions, Final State, Progress Assessment, Recommendations). Added includeSummaryInAnswer: true to ActionAgent config to enable automatic summary generation in agent answers.
Agent result display front_end/panels/ai_chat/core/AgentNodes.ts	Updated UI lane assignment logic for tool result messages: agent-tool results now unconditionally render in the agent lane, regardless of error state (previously rendered only when no error occurred).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

• AgentRunner.ts: Prompt structure and content—review correctness of the new 400–600 word multi-section format and data extraction requirements.
• ActionAgent.ts: Straightforward property addition to exported config object; minimal logic impact.
• AgentNodes.ts: Minor conditional logic refinement for lane assignment; low complexity.

Possibly related PRs

• Update agent framework structure #43: Directly related changes to AgentRunner.ts summarization flows and agent UI message handling (AgentNodes.ts/ActionAgent.ts).

Poem

🐰 ✨ Our agents now speak with eloquent grace,
Summaries detailed, no longer a race,
Configurations aligned, results on display,
In proper lane homes, come what may!
Rabbit-approved: more wisdom conveyed! 🎉

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'Fix the agent runner bug' is vague and does not clearly describe the specific changes made. While it references the AgentRunner component, it lacks specificity about what bug was fixed or what the actual changes accomplish.	Use a more descriptive title that specifies the key change, such as 'Expand agent progress summaries with detailed multi-section analysis' or 'Enable summary generation in Action Agent configuration'.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix-agent-runner-config

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ebd0883 and d9a1cee.

📒 Files selected for processing (3)

• front_end/panels/ai_chat/agent_framework/AgentRunner.ts (1 hunks)
• front_end/panels/ai_chat/agent_framework/implementation/agents/ActionAgent.ts (1 hunks)
• front_end/panels/ai_chat/core/AgentNodes.ts (1 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: claude-review

🔇 Additional comments (2)

front_end/panels/ai_chat/core/AgentNodes.ts (1)

1019-1026: Consistent agent-lane placement confirmed.

Routing every agent-tool result (even failures) into the agent lane removes the previous mismatch without affecting non-agent tools. Looks good.

front_end/panels/ai_chat/agent_framework/implementation/agents/ActionAgent.ts (1)

193-194: Flag makes ActionAgent summaries opt-in.

Setting includeSummaryInAnswer here cleanly gates the richer completion flow for this agent. No further changes needed.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Restore chat lane for agent tool errors

The new uiLane assignment now routes every configurable agent tool result to the agent lane, even when isError is true. Previously, errors were intentionally surfaced to the main chat lane so the user could see failures (see the removed comment and the existing test case shows configurable agent tool error in chat when execution fails). With this change, tool failures triggered by an agent will be hidden alongside other agent-lane messages, breaking that UX and likely failing the test that asserts errors appear in the chat lane. Consider retaining the original conditional so that only successful agent tool results stay in the agent lane while errors remain visible in the chat lane.

Useful? React with 👍 / 👎.