Enhanced Prompting Frameworks

[don-petry]

There are a variety of prompting Frameworks that people are finding as helpful in reducing complexity and cost in coding agents.

Research, analyze and consider adopting proven best practices in our Dev lead agent.

Examples:
https://github.com/DietrichGebert/ponytail
Caveman

[github-actions[bot]]

Enhancement: Enhanced Prompting Frameworks (#851)

Sharpened problem & goal

The request is to research proven prompting frameworks that reduce complexity and cost in coding agents, then selectively adopt the best patterns in the dev-lead agent's prompts.

The idea would benefit from a more specific scope: which aspect of the dev-lead agent is the target? The agent's prompts cover several distinct intents (fix-issue, fix-ci, fix-reviews, rebase, on-mention). A blanket framework adoption risks over-engineering well-tuned prompts and adding token cost. A focused audit against one or two intents first would validate the ROI before fleet-wide adoption.

Clarifying question: is the goal (a) token reduction (same quality, fewer tokens), (b) consistency (all prompts follow a common structure so they're easier to maintain), or (c) quality improvement (fewer agent errors / hallucinated plans)?

Context

The dev-lead prompt library is in prompts/dev-lead/ (fix-issue.md, fix-ci.md, fix-reviews.md, rebase.md, on-mention.md). All follow a phases-based structure (Phase 1: Scope & Plan → Phase 2: Implement → Phase 3: Verify) that already embodies some structured-prompting principles: role declaration, explicit phased execution, concrete output gates before proceeding.

The Ponytail framework provides a template system for composable, reusable prompt blocks — essentially partials/components for prompts. This maps naturally to the shared context across intents (repo standards, branch rules, auth setup) that currently repeats across the prompts/shared.md + intent-specific files. Adopting Ponytail-style composition would mean extracting repeated preamble into shared partials and assembling them at dispatch time.

"Caveman" prompting (referenced in the excerpt) typically refers to radically simplified, constraint-heavy prompting: strip the narrative, keep only imperatives and constraints. This can cut tokens but risks losing the nuanced guidance that prevents common agent mistakes (e.g. the "do not run destructive git commands" safeguards throughout fix-ci.md).

Other relevant frameworks not mentioned: COSTAR (Context, Objective, Style, Tone, Audience, Response), Chain-of-Thought with self-consistency, ReAct (Reason + Act interleaved). The dev-lead prompts already use a ReAct-adjacent pattern implicitly (plan → act → verify).

Impact / Effort

Impact: MEDIUM — the existing prompts work; the gain is maintainability (easier to update shared preamble) and potentially token savings (less repetition).

Effort: M — requires reading and scoring each prompt file against candidate frameworks, running evals before/after on the held-out eval set (evals/), and updating the Ponytail/component infrastructure if adopted.

Suggested acceptance criteria

• A written audit of all 5 dev-lead prompt files (fix-issue.md, fix-ci.md, fix-reviews.md, rebase.md, on-mention.md) scores each against at least one named framework (Ponytail, COSTAR, or Caveman), identifying the top 3 structural improvements.
• Any adopted framework is piloted on a single intent (suggested: fix-issue.md — highest token spend, most structured) and validated against the held-out eval set in evals/ before touching other intents.
• Token count per dev-lead run is captured via the existing TOKEN_LOG_FILE / token-report infrastructure before and after, and shows no regression (ideally a reduction).
• Shared preamble (role declaration, repo-standards context, git-safety rules) is extracted into a single composable partial and referenced by all intent prompts — no copy-paste repetition.
• Any removed "narrative" guidance that previously prevented a known agent failure mode is replaced by an explicit constraint (not silently dropped).