Skip to content

Warn when plan agent model refuses to tool-call #652

@anandgupta42

Description

@anandgupta42

Problem

Telemetry analysis of 2026-04-04 showed the `plan` agent has a 31% completion rate. Every abandoned plan session has the same signature: one generation, `finish_reason=stop`, zero tool calls.

Root cause: some models (observed: `ollama-cloud/qwen3-coder-next` with 5/5 failures, occasionally `openai/gpt-5.4`) respond to the plan agent's system prompt with text instead of tool calls. The user sees a text answer, no exploration happening, and aborts the session.

Representative telemetry burst (machine `450185a5`, 2026-04-04):

  • 5 plan sessions with `qwen3-coder-next` → all `stop`, all abandoned in 10-38s
  • 6th session switches to `minimax-m2.7` → immediately tool-calls, completes in 97s

Proposed Fix

When the plan agent finishes a step with `finish_reason=stop` AND no tool calls have ever been made in the session, show a one-shot warning:

  • `log.warn` with session/model/provider context
  • Emit `plan_no_tool_generation` telemetry event (per-model abandonment tracking)
  • Inject a synthetic text part visible in the TUI recommending the user try a different model via `/model`

Synthetic text parts are filtered from LLM context replay, so this doesn't pollute future turns.

Acceptance Criteria

  • Warning fires exactly once per session
  • Only fires for `agent=plan`, `finish_reason=stop`, `sessionToolCallsMade=0`
  • Other finish_reasons (`tool-calls`, `length`, `error`) don't trigger
  • Other agents (`builder`, `analyst`, `explore`, `general`) don't trigger
  • New telemetry event type `plan_no_tool_generation` with model_id, provider_id, tokens_output

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions