plan action hallucinates field absence; add "verify before asserting existence" rule to PLAN_AGENT_PROMPT

[monkut]

Summary

The plan action's PLAN_AGENT_PROMPT (askcc/definitions.py:124) instructs the agent to "Read relevant source files... Do not speculate about unopened code." This guidance is too soft to prevent the agent from claiming a field/symbol does not exist when it actually does. As a result, generated implementation plans can include AddField migration steps for columns that already exist — which would either fail at apply time or silently corrupt the model definition if implemented as written.

Reproduction

askcc -i plan --github-issue-url https://github.com/kiconiaworks/kippo/issues/7

The generated Implementation Plan included:

Note: display_in_project_report field — mentioned in the issue's "set close fields" and "hide" lists but does not currently exist in the model. Will add display_in_project_report = models.BooleanField(_("Display in Project Report"), default=True) ...

In reality the field exists at kippo/projects/models.py:206:

display_in_project_report = models.BooleanField(
    _("Display in Project Report Summary (slack)"),
    default=True,
    help_text=_("If True, project will be included in the Project Report Summary"),
)

A simple grep -n "display_in_project_report" kippo/projects/models.py would have surfaced this immediately, but the agent did not run it before writing the absence claim.

Impact if implemented as-written

• Duplicate Python attribute declaration — one of the two definitions wins silently, potentially altering verbose_name / help_text.
• The auto-generated AddField migration fails at apply time: column "display_in_project_report" of relation "projects_kippoproject" already exists.
• Misleading "Confirmation requested" question to the user about a non-decision.

Proposed fix

Update PLAN_AGENT_PROMPT in askcc/definitions.py to add an explicit verification rule and tighten the "Current state" must-include item:

 Read relevant source files, tests, and config before planning. Do not speculate about unopened code.
 
+Verify before asserting existence: for every symbol you propose to add (model field,
+function, class, setting, file), first `grep -n '<name>' <target_file>` or Read the
+target file to confirm absence. Never write "<symbol> does not currently exist" or
+propose `AddField` / "create new <file>" steps based on issue text alone. If grep
+finds the symbol, the step becomes "modify existing", and any migration must NOT
+include `AddField` for an already-present column.
+
 Your plan must include:
-1. Current state — what exists today related to the issue.
+1. Current state — what exists today related to the issue. Cite `file:line` for each
+existing symbol referenced; for any symbol you claim is missing, cite the negative
+grep result (e.g. `grep -n 'foo' models.py → no match`).
 2. Step-by-step implementation tasks, each referencing specific files and functions.

Acceptance criteria

• PLAN_AGENT_PROMPT includes the "Verify before asserting existence" paragraph requiring grep/Read before any " does not exist" claim or AddField / "create new" step.
• PLAN_AGENT_PROMPT "Current state" item requires file:line citation for existing references and negative-grep result for absence claims.
• Re-running the plan action against an issue that mentions an existing symbol does not produce an AddField step for that symbol (regression test target).

Related

• Source incident: kippo projectsのclose時に"upsell action"を追加できるようにする kiconiaworks/kippo#7 (Implementation Plan section). The incorrect plan has since been corrected manually.