Skip to content

fix: [#652] warn when plan agent model refuses to tool-call#653

Merged
anandgupta42 merged 1 commit intomainfrom
fix/plan-agent-no-tool-warning
Apr 5, 2026
Merged

fix: [#652] warn when plan agent model refuses to tool-call#653
anandgupta42 merged 1 commit intomainfrom
fix/plan-agent-no-tool-warning

Conversation

@anandgupta42
Copy link
Copy Markdown
Contributor

@anandgupta42 anandgupta42 commented Apr 5, 2026

What does this PR do?

Warns the user when the plan agent finishes a step with finish_reason=stop and zero tool calls in the session — catching models that refuse to tool-call in plan mode (observed on ollama-cloud/qwen3-coder-next, occasionally openai/gpt-5.4).

Signals emitted (one-shot per session):

  • log.warn with session/model/provider context
  • plan_no_tool_generation telemetry event (new type) — per-model abandonment tracking
  • Synthetic text part in the TUI with a /model recommendation

Guards:

  • Fires at most once per session (planNoToolWarningEmitted flag)
  • Only for agent="plan", finish_reason="stop", sessionToolCallsMade===0
  • Other finish reasons (tool-calls, length, error) don't trigger
  • Other agents (builder, analyst, explore, general) don't trigger

The text part is synthetic: true — shown to the user in the TUI but excluded from the transcript replayed to the LLM next turn (existing filter at prompt.ts:648, 795).

Type of change

  • Bug fix
  • New feature
  • Test coverage
  • Documentation
  • Refactoring
  • Infrastructure

Issue for this PR

Closes #652

How did you verify your code works?

  • Typecheck: clean on changed files (existing config.ts errors unchanged)
  • Marker guard: passes (altimate_change markers around all additions)
  • 6 new unit tests in processor.test.ts covering:
    • happy path (fires on plan + stop + no-tools)
    • negative: already tool-called this session
    • negative: finish_reason=tool-calls
    • negative: non-plan agents (builder/analyst/general/explore)
    • negative: finish_reason=length/error/content-filter/unknown
    • one-shot flag (doesn't re-fire)
  • All 46 processor tests pass

Checklist

  • Typecheck passes
  • Marker guard passes
  • No breaking changes (additive telemetry event + synthetic warning)
  • Telemetry event typed and documented
  • Unit tests for all detection branches

Summary by cubic

Warns when the plan agent stops without any tool calls so users aren’t stuck and we can track model refusals. Addresses #652.

  • Bug Fixes
    • Trigger: agent="plan", finish_reason=stop, and no session tool calls.
    • Actions: emit log.warn; send plan_no_tool_generation telemetry (with model/provider and tokens_output); add a synthetic TUI note suggesting /model.
    • Guards: one-shot per session; ignored for other agents or finish reasons.
    • Tests: 6 new cases cover positive/negative paths; all processor tests pass.

Written for commit 31db2ed. Summary will update on new commits.

Summary by CodeRabbit

New Features

  • Plan agent now displays a warning when it completes without generating any tool calls, providing visibility into agent behavior during sessions.

Some models (observed: ollama-cloud/qwen3-coder-next, occasionally
openai/gpt-5.4) end plan-agent steps with `finish_reason=stop` and
never emit tool calls. Users see a text answer, no exploration, and
abandon the session — accounting for most of the plan agent's 31%
completion rate seen in 2026-04-04 telemetry.

When the plan agent finishes a step with `stop` AND no tool calls have
been made in the session, fire a one-shot warning:
- `log.warn` with session/model/provider context
- `plan_no_tool_generation` telemetry event (per-model tracking)
- Synthetic text part in the TUI recommending `/model` switch

The text part is `synthetic: true` so it's shown to the user but
excluded from the transcript sent back to the LLM next turn.

Closes #652
Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 5, 2026

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: d18e5bbf-4057-49fd-89e8-ee1f7113f5b8

📥 Commits

Reviewing files that changed from the base of the PR and between ba9882f and 31db2ed.

📒 Files selected for processing (3)
  • packages/opencode/src/altimate/telemetry/index.ts
  • packages/opencode/src/session/processor.ts
  • packages/opencode/test/session/processor.test.ts
 __________________________________________________
< Your stack overflowed; I brought a bigger stack. >
 --------------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)

✅ Unit Test PR creation complete.

  • Create PR with unit tests
  • Commit unit tests in branch fix/plan-agent-no-tool-warning

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 5, 2026

Note

Unit test generation is a public access feature. Expect some limitations and changes as we gather feedback and continue to improve it.


Generating unit tests... This may take up to 20 minutes.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

@anandgupta42 anandgupta42 merged commit 478d893 into main Apr 5, 2026
20 checks passed
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 5, 2026

✅ Created PR with unit tests: #654

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Warn when plan agent model refuses to tool-call

1 participant