Skip to content

Conversation

@ThomasK33
Copy link
Member

This tightens Plan Mode guidance so agents:

  • use ask_user_question for unresolved questions (instead of an “Open Questions” section / inline questions)
  • avoid duplicating plan content and plan file paths after calling propose_plan (the Plan UI already renders both)

Validation:

  • make static-check

đź“‹ Implementation Plan

Reinforce ask_user_question usage when the agent has open questions (Plan Mode)

Goal

Ensure that in Plan Mode, if the model has unanswered questions that materially affect the plan, it uses the ask_user_question tool (instead of outputting an “Open questions” section or asking inline in chat).

Recommended approach (A): tighten the Plan Mode system instruction (primary)

Why here: getPlanModeInstruction() is injected into the system message for every Plan Mode turn, so it’s the most reliable place to enforce behavior across models.

Change: update src/common/utils/ui/modeUtils.ts → getPlanModeInstruction() to add an explicit “no open questions in the plan; use tool” rule.

Proposed wording to insert (exact text; adjust line wraps as needed):

If you need clarification from the user before you can finalize the plan, you MUST use the ask_user_question tool.
- Do not ask questions in a normal chat message.
- Do not include an "Open Questions" section in the plan.
- Ask up to 4 questions at a time (each with 2–4 options; "Other" is always available for free-form input).
- After you receive answers, update the plan file and only then call propose_plan.
- After calling propose_plan, do not repeat/paste the plan contents in chat; the UI already renders the full plan.
- After calling propose_plan, do not say “the plan is ready at <path>” or otherwise mention the plan file location; it’s already shown in the Plan UI.

Net LoC (product code): +8 to +15 (string expansion only).

Reinforcement (B): strengthen the tool description (secondary)

Why: tool descriptions are a strong hint for many models, and this also helps when users add Tool: ask_user_question overrides (they’ll see a better base description).

Change: update src/common/utils/tools/toolDefinitions.ts → ask_user_question.description.

Proposed wording tweak (replace “should be used…” with a requirement + anti-pattern callout):

This tool is intended for plan mode and MUST be used when you need user clarification to complete the plan.
Do not output a list of open questions; ask them via this tool instead.

Net LoC (product code): ~0 to +3 (string edit).

Tests

Add small unit tests so this behavior doesn’t regress during refactors.

  1. src/common/utils/ui/modeUtils.test.ts

    • Assert that getPlanModeInstruction("/tmp/plan.md", false) contains ask_user_question and MUST.
    • Assert that it includes the “don’t repeat/paste plan contents” + “don’t mention plan file location” guidance (to prevent UI clutter).
  2. src/common/utils/tools/toolDefinitions.test.ts

    • Import the tool definitions map and assert ask_user_question.description contains the “MUST be used …” phrasing.

Net LoC (product code): 0 (tests only).

Optional docs follow-up (not required for behavior)

If we want the user-facing docs to match the new behavior:

  • docs/plan-mode.mdx: change “may call ask_user_question …” → “should/must call … when it needs clarification before finalizing a plan.”

Net LoC (product code): 0 (docs only).

Validation checklist (Exec Mode)

  • Run make typecheck.
  • Run targeted unit tests:
    • bun test src/common/utils/ui/modeUtils.test.ts
    • bun test src/common/utils/tools/toolDefinitions.test.ts
  • Manual sanity check in mux:
    • Enter Plan Mode and ask for a plan on an underspecified task.
    • Confirm the agent calls ask_user_question instead of emitting an “Open questions” section.

Alternative considered: backend enforcement / linting

Idea: detect “Open Questions” patterns in the assistant output and block propose_plan unless ask_user_question was called.

Why not (for now):

  • Hard to do safely without false positives/negatives.
  • The backend can’t reliably synthesize the multiple-choice options the tool requires.

This can be revisited if prompt-only enforcement proves insufficient.

Net LoC (product code): ~50–150.


Generated with mux • Model: openai:gpt-5.2 • Thinking: xhigh

@chatgpt-codex-connector
Copy link

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

- Require ask_user_question for plan clarification (no inline/open-question lists)
- Reduce post-propose_plan chatter (no plan paste + no file-path mention)
- Strengthen tool descriptions and add unit coverage

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with `mux` • Model: openai:gpt-5.2 • Thinking: xhigh_
<!-- mux-attribution: model=openai:gpt-5.2 thinking=xhigh -->

Change-Id: I1f9e6e8a373f528c6d47057cbcecc2ce6fe16886
@ThomasK33 ThomasK33 force-pushed the reinforce-ask-user-question-tool branch from 778d606 to cca17b5 Compare December 15, 2025 10:37
@ThomasK33 ThomasK33 added this pull request to the merge queue Dec 15, 2025
Merged via the queue into main with commit 107ce63 Dec 15, 2025
59 of 65 checks passed
@ThomasK33 ThomasK33 deleted the reinforce-ask-user-question-tool branch December 15, 2025 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant