Workflow: Harden kelos-workers prompt to reduce closed-without-merge rate #508
Description
🤖 Kelos Agent @gjkim42
Summary
The kelos-workers TaskSpawner prompt (self-development/kelos-workers.yaml) is missing several instructions that would prevent recurring failure patterns observed in recent agent PRs. Analysis of the last 20 agent PRs shows 6 out of 15 were closed without merge — a 40% rejection rate indicating systemic prompt gaps.
Evidence
Pattern 1: Scope creep — agents add unrequested features
PR #493 (closed): Issue #182 asked for client-side validation of --type and --credential-type flags. The agent also added custom as a new agent type, which was not part of the issue scope. Maintainer feedback: "Do we have server side validation for this? If it doesn't create an issue for it. (Also custom agent should be allowed.)" — the maintainer wanted the agent to investigate the broader system before implementing, and to file a separate issue for scope beyond the original request.
Fix: Add an explicit instruction to only implement what the issue asks for, and to create follow-up issues for related improvements discovered during investigation rather than bundling them into the PR.
Pattern 2: Not reading linked issues or prior context on reset
PR #444 (closed after 5 /reset-worker rounds): The maintainer repeatedly told the agent to read issue #456 (axon→kelos rename). The agent failed to do so across 4 resets. Each reset starts the agent fresh without prior review context.
Fix: Add an instruction in the "If a PR already exists" path to also read all comments on the issue (not just the PR), since maintainer guidance may be posted there. The current prompt only says Read ALL review comments and conversation on the PR.
Pattern 3: Not searching codebase for existing patterns before creating new ones
PR #493: The automated reviewer flagged that the agent created duplicate validAgentTypes and validCredentialTypes lists when these constants already existed elsewhere in the codebase. This is a P1 duplication issue.
Fix: Add a general instruction to search the codebase for existing constants, types, and helper functions before creating new ones.
Pattern 4: PR description format causes downstream workflow failures
Issue #499 documents this: the /squash-commits workflow requires PRs to have a properly formatted closing reference (e.g., Fixes #123). The worker agent sometimes writes the issue number in natural language (e.g., Fixes the ordering (#461)) which the regex cannot parse.
Fix: Add an explicit instruction specifying the required PR description format: the body must contain Fixes #<number> or Closes #<number> on its own line, matching the standard GitHub closing keyword format.
Pattern 5: Design disagreements on re-attempt
PRs #470 and #486 (both closed): Two attempts at the same feature (--crd flag) with incorrect defaults. The maintainer's feedback on #470 was "Is it better to just install crds if it's provided?" but the agent re-submitted #486 with the same design 4 hours later, which was closed within 5 minutes.
Fix: In the "If a PR already exists" path, add an instruction that when a previous PR for the same issue was closed, the agent must read the closed PR's reviews to understand why it was rejected before starting a new approach.
Proposed changes to kelos-workers.yaml promptTemplate
Add the following instructions:
After "2b. Investigate the issue" (new PR path):
- Read ALL comments on the issue, including maintainer feedback and linked issues/PRs.
- If previous PRs for this issue were closed, read their reviews to understand why before choosing your approach.
- Search the codebase for existing constants, types, and patterns before creating new ones. Do not duplicate definitions.
- Only implement what the issue explicitly asks for. If you discover related improvements, create separate issues for them.
After "2a. Read ALL review comments" (existing PR path):
- Also read ALL comments on the linked issue (gh issue view <number> --comments), not just the PR.
In the PR creation/update steps:
- The PR body MUST contain a standard closing keyword reference on its own line (e.g., `Fixes #<number>` or `Closes #<number>`). Do not embed the issue number in natural language.
Impact
These changes address the root causes of 5 out of 6 recently rejected PRs. The remaining rejection (#487, cosmetic README change) was a low-value change that the existing "keep changes minimal" instruction in agentconfig.yaml should already prevent, but the worker prompt doesn't reinforce this.
Related issues
- Enforce kelos-worker PRs to include proper closing issue reference #499 — PR closing reference format (overlaps with Pattern 4 above; this issue addresses it at the prompt level)
- Workflow: Triage-to-worker handoff loses structured context — add WorkItem metadata enrichment #455 — Triage-to-worker handoff context loss (complementary; this issue improves what the worker does with the context it has)