PDD bug accumulates unnecessary test files

[niti-go]

Problems

1. pdd bug Step 9 always runs and creates E2E test files, even if the bug may not warrant an E2E test. The unit test from Step 7 might already provide full coverage for simple internal bugs, but Step 9 would still run and create a new test file that adds no value.
2. pdd bug Step 6's prompt presents two options to the LLM: append to an existing test file (append) or create a new test file (new). However, in practice, the agent almost always picks (new) because the prompt portrays them as equally valid options, and the example reinforces (new) as the safer choice.

Both of these problems can cause the /tests directory to grow unnecessarily by adding many small issue-specific files.

Proposed fixes

1. Before step 9, add a classification check: "Is there a user-facing behavior not already covered by the unit test?" Only proceed to E2E generation if yes.
2. Adjust step 6's prompt to specify that appending to the nearest existing test file is the preferred default. The agent should only create a new file when no relevant test file exists. Also, update the example in the prompt to reflect this.

[prompt-driven-github]

🚀 Job Queued!

Job ID: f5zKQsQTPhqFz7D1qcLQ
Triggered by: @jiaminc-cmu
Label: enhancement

View Live Progress
Model: pdd-opus (claude-opus-4-6)

[prompt-driven-github]

Step 1: Duplicate Check

Status: Related issues found, but not exact duplicates — proceeding

Search Performed

• Searched for: "test files bug accumulate", "E2E test step 9", "pdd bug test", "append test file", "test depth"
• Issues reviewed: ~40 results across 3 searches, 4 issues examined in detail

Findings

Two open issues overlap with parts of this request:

Issue #102 — "pdd bug + pdd test incremental to increase coverage should automatically append tests to test file"

• Overlap: Directly relates to Problem LiteLLM: AttributeError: 'Cache' object has no attribute 'cache' #2 in this issue (Step 6 creating new files instead of appending to existing ones)
• Status: OPEN, unresolved. Has root cause analysis and implementation progress in comments, but no merged fix.
• Difference: pdd bug + pdd test incremental to increase coverage should automatically append tests to test file #102 focuses on pdd sync/pdd test append behavior broadly; PDD bug accumulates unnecessary test files #642 specifically targets Step 6's prompt wording and example bias.

Issue #558 — "pdd bug should create tests according to the depth of the issue"

• Overlap: Relates to Problem pdd install_completion: install_completion() got unexpected keyword argument 'quiet' #1 in this issue (Step 9 always running E2E tests regardless of bug complexity)
• Status: OPEN, unresolved. Minimal activity (only deadline comments, no implementation).
• Difference: pdd bug should create tests according to the depth of the issue #558 is broader about test depth; PDD bug accumulates unnecessary test files #642 proposes a specific classification gate before Step 9.

Other reviewed issues:

• pdd bug generates structural tests (param existence checks) instead of behavioral tests #507 — About structural vs behavioral test quality, not about file accumulation. Not a duplicate.
• PDD-CLI Bug: Generates Features Without Corresponding Test Coverage #589 (CLOSED) — About features lacking tests entirely. Opposite problem. Not a duplicate.

Decision

Neither #102 nor #558 has been resolved, and #642 combines both concerns with more specific proposed fixes (classification check before Step 9, prompt rewording in Step 6). This issue should proceed as a new change request that supersedes the unresolved portions of #102 and #558.

---

Proceeding to Step 2: Documentation Check

[prompt-driven-github]

Step 2: Documentation Check

Status: Not Implemented

Documentation Reviewed

• pdd/prompts/agentic_bug_step6_test_plan_LLM.prompt — Step 6 test plan prompt
• pdd/prompts/agentic_bug_step7_generate_LLM.prompt — Step 7 test generation prompt
• pdd/prompts/agentic_bug_step9_e2e_test_LLM.prompt — Step 9 E2E test prompt
• pdd/prompts/agentic_bug_orchestrator_python.prompt — Orchestrator spec
• pdd/agentic_bug_orchestrator.py — Orchestrator implementation
• README.md, CONTRIBUTING.md

Findings

Problem 1 — No classification gate before Step 9:
The orchestrator (agentic_bug_orchestrator.py:405-609) runs Step 9 unconditionally for every bug. While the Step 9 prompt includes an E2E_SKIP escape hatch (line 295: "If no E2E test is applicable... output E2E_SKIP"), there is no pre-Step-9 classification check that evaluates whether user-facing behavior is already covered by the unit test. The agent always enters Step 9, consuming time and cost, and must self-determine whether to skip. The issue requests a dedicated classification step before Step 9 runs.

Problem 2 — Step 6 prompt bias toward creating new test files:
In agentic_bug_step6_test_plan_LLM.prompt:106-107, the Test Location example presents both options as equally valid:

**File:** `tests/test_module.py` (append) or `tests/test_module_bug_123.py` (new)

There is no guidance that appending to an existing test file should be the preferred default. The prompt does not instruct the agent to only create a new file when no relevant test file exists. This leads the agent to almost always pick (new), accumulating unnecessary test files.

Neither proposed fix from the issue is currently implemented.

---

Proceeding to Step 3: Identify Dev Units

[prompt-driven-github]

Step 3: Research & Clarification

Status: Research Complete

Summary

Researched best practices for test file organization (append vs. new) and conditional step execution in LLM agent workflows. Both proposed fixes align well with established patterns and can be implemented with minimal changes to the existing orchestrator and prompt infrastructure.

Research Findings

Technical Background

1. Test file organization: The widely accepted convention is "one test file per source module." New test cases for the same module should be appended to the existing test file. Creating a new file is appropriate only when testing a genuinely new component or when an existing file has grown unwieldy.
2. Conditional step execution in LLM agents: Classification gates are a standard pattern in agent orchestration frameworks (LangGraph, OpenAI agent patterns). A gate function evaluates prior step outputs and returns a routing decision (proceed/skip), preventing unnecessary work. PDD already uses this pattern — Step 5.5 (prompt classification) is a classification gate that routes between code fixes and prompt fixes.

Best Practices

• Append-first for tests: Industry consensus is that tests should mirror source structure. A bug in foo.py should add tests to test_foo.py, not create test_foo_issue_642.py. New files only when no relevant test file exists.
• Lightweight classification gates: The gate should be a simple LLM classification prompt (not a full agent step) that outputs a structured decision. The existing E2E_SKIP escape hatch inside Step 9 is insufficient because it still incurs the full cost of spinning up the step (2000s timeout, environment discovery, etc.).
• Explicit default in prompts: When presenting options to an LLM, the option shown first with reinforcing language ("preferred", "default") is chosen far more often. The current Step 6 prompt presents (append) and (new) as equals, which biases toward (new) because the example shows (new).

Similar Implementations

• PDD's own Step 5.5: Already implements a classification gate pattern. It classifies the defect type (code vs. prompt) before proceeding. The proposed Step 8.5 gate can follow the exact same pattern — a lightweight classification prompt that outputs a structured signal (e.g., E2E_NEEDED: yes/no).
• LangGraph conditional edges: Use "gate functions" that check conditions from prior state and route to different nodes. This is essentially what the orchestrator's post-step logic already does (checking for E2E_SKIP:, Duplicate of #, etc.).

Security/Edge Cases

• No security concerns: Both changes are prompt text modifications and orchestrator control flow — no new attack surface.
• Edge case — false negatives on the gate: The classification gate might incorrectly skip E2E tests for bugs that do have user-facing impact. Mitigation: bias the gate prompt toward "yes, run E2E" when uncertain (err on the side of running E2E rather than skipping).
• Edge case — no existing test file to append to: Step 6 prompt must retain (new) as a valid fallback for when no relevant test file exists in the project.
• Edge case — existing test file is very large: The append-first guidance should include a size/relevance check — if the nearest test file is unrelated or excessively large, creating a new file is still appropriate.

Refined Specifications

Based on the research, here are clarified requirements:

1. Problem 1 fix — Add Step 8.5 classification gate: Insert a lightweight classification step between Step 8 and Step 9 in the orchestrator. It should receive the issue content, root cause analysis (Step 5), and unit test output (Step 7/8), then output E2E_NEEDED: yes or E2E_NEEDED: no with a one-line rationale. If no, skip Step 9 entirely (don't invoke the agent at all). Follow the Step 5.5 pattern for implementation.
2. Problem 2 fix — Reword Step 6 prompt: In the "Plan test location" section and "Test Location" output example, make (append) the explicit default. Change the example from tests/test_module_bug_123.py (new) to tests/test_module.py (append). Add guidance: "Prefer appending to the nearest existing test file for the affected module. Only create a new file when no relevant test file exists."
3. Edge case handling: The classification gate prompt should default to E2E_NEEDED: yes when uncertain (conservative). The Step 6 prompt should retain (new) as a valid option but clearly mark it as the exception, not the default.

Alternative Approaches Considered

• Inline the gate into Step 9's prompt (no new step): Pros: no orchestrator changes needed, the existing E2E_SKIP escape hatch already supports this. Cons: still incurs the full 2000s timeout and agent spin-up cost; the issue explicitly asks to avoid running the step entirely. Not recommended.
• Use a deterministic heuristic instead of LLM classification: Pros: faster, cheaper, no LLM call needed. Cons: hard to define a reliable heuristic for "user-facing behavior" — the LLM is better suited for this semantic judgment. Not recommended as sole approach, but could be used as a fast pre-filter (e.g., if root cause mentions only internal/private functions, skip E2E without LLM call).
• Remove Step 9 entirely and rely only on unit tests: Pros: simplest fix. Cons: E2E tests do add value for user-facing bugs (CLI behavior, API responses). Not recommended.

Recommended Approach

Implement both fixes as proposed in the issue:

1. Add a new Step 8.5 classification prompt (following the Step 5.5 pattern) with orchestrator logic to skip Step 9 when E2E_NEEDED: no. This is a clean separation of concerns — classification logic stays in the prompt, routing logic stays in the orchestrator.
2. Reword Step 6's prompt to make (append) the preferred default, updating both the guidance text and the example output. This is a straightforward prompt edit with high expected impact on agent behavior.

Both changes are low-risk, well-precedented within the codebase, and directly address the root causes identified in the issue.

---

Proceeding to Step 4: Documentation Changes

[prompt-driven-github]

Step 4: Requirements Clarity Check

Status: Requirements Clear

Clarity Assessment

Aspect	Status
Problem statement	✅ Clear
Expected outcomes	✅ Defined
Scope	✅ Bounded
Edge cases	✅ Addressable

Understanding Summary

Two changes to the pdd bug workflow: (1) Add a Step 8.5 classification gate that checks whether E2E testing is needed before running Step 9, following the existing Step 5.5 pattern. (2) Reword Step 6's prompt to make (append) the explicit default for test file placement and update the example accordingly.

Assumptions Made

• The Step 8.5 gate will be a lightweight LLM classification prompt (consistent with the Step 5.5 precedent) that outputs E2E_NEEDED: yes/no, and the orchestrator will conditionally skip Step 9 when the answer is no.
• For Step 6, "nearest existing test file" means the most relevant test file in the same module/directory as the code being tested — the agent will still be allowed to create a new file when no relevant test file exists.
• The E2E_SKIP escape hatch already in Step 9's prompt can remain as a secondary safeguard, but the primary gate will be the new Step 8.5 check.

---

Proceeding to Step 5: Documentation Changes

[prompt-driven-github]

Step 5: Documentation Changes

Status: Documentation Updates Required

Summary

Two sections of README.md need updates to reflect the new Step 8.5 classification gate and the append-first preference in Step 6. No new documentation files are needed — both changes are internal workflow refinements already described in the existing pdd bug command reference.

Files to Update

README.md

Section: pdd bug workflow steps (line ~2269)
Change: Update Step 6 description to mention append-first preference

Current (line 2269):

6. **Test plan** - Design a plan for creating tests to detect the problem. Enumerates all affected output channels to ensure complete coverage. Posts comment with the test plan.

Draft replacement:

6. **Test plan** - Design a plan for creating tests to detect the problem. Enumerates all affected output channels to ensure complete coverage. Prefers appending to the nearest existing test file; only creates a new test file when no relevant file exists. Posts comment with the test plan.

---

Section: pdd bug workflow steps (line ~2275)
Change: Add Step 8.5 and update Step 9 description

Current (line 2275):

9. **E2E test** - Generate and run end-to-end tests to verify the bug at integration level. May be skipped when the LLM determines unit tests provide sufficient coverage. Posts comment with E2E test results or skip reason.

Draft replacement:

8.5. **E2E classification** - Classify whether a user-facing behavior exists that is not already covered by the unit test. Only proceeds to Step 9 if E2E testing is warranted. Posts comment with classification result (`E2E_NEEDED: yes/no`).

9. **E2E test** - Generate and run end-to-end tests to verify the bug at integration level. Skipped when Step 8.5 determines unit tests provide sufficient coverage. Posts comment with E2E test results or skip reason.

No Changes Needed

• docs/prompting_guide.md — References pdd bug only in the Workflow Cheatsheet table; the table describes the high-level flow (create failing test → fix) which remains accurate.
• docs/whitepaper.md — No pdd bug step-level references.
• docs/TUTORIALS.md — Shows pdd bug usage examples only; no step-level detail to update.
• docs/ONBOARDING.md — Shows pdd bug usage examples only; no step-level detail to update.
• docs/faq.md — No relevant pdd bug workflow content.
• CONTRIBUTING.md — No pdd bug workflow content.
• CHANGELOG.md — Will be updated at release time, not as part of this change.

---

Proceeding to Step 6: Identify Dev Units

[prompt-driven-github]

Step 6: Dev Units Identified

Status: Found 3 dev units (2 MODIFY, 1 CREATE)

Change Summary

Two changes to the pdd bug workflow: (1) Add a Step 8.5 classification gate (E2E_NEEDED: yes/no) before Step 9 to skip unnecessary E2E test generation, and (2) Reword Step 6's prompt to make appending to existing test files the default over creating new ones.

Project Structure (from .pddrc)

• Source directory: pdd/
• Test directory: tests/
• Example directory: context/
• Language: python

Dev Units to MODIFY

Prompt	Code	Example	Test
pdd/prompts/agentic_bug_orchestrator_python.prompt	pdd/agentic_bug_orchestrator.py	context/agentic_bug_orchestrator_example.py	tests/test_agentic_bug_orchestrator.py
pdd/prompts/agentic_bug_step6_test_plan_LLM.prompt	(LLM prompt — no code file)	(none)	(none)

Reasons for MODIFY:

1. agentic_bug_orchestrator_python.prompt — Two changes needed:

   • Problem 1 (E2E gate): Add Step 8.5 to the step sequence table and step definitions. Currently the orchestrator defines steps [1, 2, 3, 4, 5, 5.5, 6, 7, 8, 9, 10]. A new step (8.5, "e2e_classification", "Classifying whether E2E test is needed") must be inserted between steps 8 and 9. The orchestrator must parse the Step 8.5 output for E2E_NEEDED: no and skip Step 9 if present (similar to how Step 5.5 prompt classification works). Update step count from 11 to 12 in display labels. Add timeout for step 8.5 to BUG_STEP_TIMEOUTS. Update context accumulation docs.
   • Problem 2: No orchestrator changes needed for the append-vs-new fix (that's purely in the Step 6 LLM prompt).

2. agentic_bug_step6_test_plan_LLM.prompt — Problem 2 fix:

   • In section "4. Plan test location" (lines 66-70), the current text presents (append) and (new) as equally valid options with no stated preference.
   • In the output template (line 106), the example shows (append) or ... (new) as equal choices.
   • Fix: Reword to make (append) the explicit default. The agent should only create a new file when no relevant existing test file can be found. Update the example in the output template to show (append) as the primary example.

Dev Units to CREATE

Prompt	Code	Example	Test
pdd/prompts/agentic_bug_step8_5_e2e_classification_LLM.prompt	(LLM prompt — no code file)	(none)	(none)

Reuse Analysis (REQUIRED for each CREATE)

• Module: agentic_bug_step8_5_e2e_classification_LLM
• Existing modules considered:
  • agentic_bug_step9_e2e_test_LLM.prompt — Already has an E2E_SKIP escape hatch (line 145, 155, 295), but this fires after Step 9 starts (consuming the full 2000s timeout). The issue requests a pre-step classification to avoid spinning up Step 9 at all.
  • agentic_bug_step5_5_prompt_classification_LLM.prompt — This is the pattern to follow (lightweight classification step inserted between existing steps). Cannot be extended to also classify E2E need since it serves a completely different purpose (code bug vs prompt defect).
  • agentic_bug_step8_verify_LLM.prompt — Step 8 verifies the unit test works. Could theoretically add E2E classification here, but that would conflate two concerns (verification + classification) and make the step harder to maintain independently.
• Why extension not possible: No existing step performs pre-E2E classification. The Step 9 E2E_SKIP escape hatch is insufficient because it only triggers after the expensive step has already started. A new lightweight prompt (following the Step 5.5 pattern) is the correct approach per the research in Step 3.

Dependencies (may need interface updates)

• pdd/prompts/agentic_bug_python.prompt — Depends on agentic_bug_orchestrator. The step count description ("10-step investigation process" on line 7) should be updated to reflect the new step 8.5 (now 12 steps). This is a minor documentation-level change in the prompt's Role & Scope section.

Integration Points Discovered

Integration Point	Reason for Update	Prompt File
pdd/agentic_bug_orchestrator.py	New step 8.5 must be added to steps list (line 387), BUG_STEP_TIMEOUTS dict (line 45), and post-step parsing logic (after line 581)	pdd/prompts/agentic_bug_orchestrator_python.prompt

Search performed:

• Searched pdd/agentic_bug_orchestrator.py for step definitions, STEP_TIMEOUTS, and step 9 handling logic
• Searched pdd/agentic_common.py for shared timeout constants (none found — timeouts are in orchestrator)
• Searched all pdd/prompts/agentic_bug_step*.prompt files for the complete step chain

Frontend Dev Units (Cross-Layer)

Backend endpoints being created/modified:

• No API endpoints affected. This change modifies internal agentic workflow steps (LLM prompts and orchestrator logic), not user-facing APIs.

Frontend files referencing these endpoints:

• N/A — no cross-layer impact.

Direct Edit Candidates (No Prompt)

• None identified. All changes are to PDD prompt files or PDD-generated code.

Files Explored

• .pddrc — project structure configuration
• pdd/prompts/agentic_bug_orchestrator_python.prompt — orchestrator prompt (MODIFY)
• pdd/prompts/agentic_bug_step6_test_plan_LLM.prompt — Step 6 prompt (MODIFY)
• pdd/prompts/agentic_bug_step9_e2e_test_LLM.prompt — Step 9 prompt (reference for E2E_SKIP pattern)
• pdd/prompts/agentic_bug_step5_5_prompt_classification_LLM.prompt — Step 5.5 prompt (pattern reference for classification gate)
• pdd/prompts/agentic_bug_step8_verify_LLM.prompt — Step 8 prompt (checked for extension possibility)
• pdd/prompts/agentic_bug_python.prompt — CLI entry point prompt (dependency)
• pdd/agentic_bug_orchestrator.py — generated orchestrator code (lines 377-610, step definitions and post-step logic)
• pdd/agentic_common.py — shared agentic utilities
• context/agentic_bug_orchestrator_example.py — orchestrator example
• context/agentic_bug_example.py — bug entry point example
• tests/test_agentic_bug_orchestrator.py — orchestrator tests
• tests/test_agentic_bug.py — bug entry point tests
• README.md — documentation (searched for pdd bug workflow description)

---

Proceeding to Step 7: Architecture Review

[prompt-driven-github]

Step 7: Architecture Review

Status: Architecture Clear

Reuse Validation

• agentic_bug_step8_5_e2e_classification_LLM.prompt (CREATE) — Confirmed necessary. The existing E2E_SKIP escape hatch inside Step 9's prompt runs after the step has already started (incurring the 2000s timeout cost). A lightweight pre-step classification gate avoids spinning up Step 9 entirely for simple internal bugs. This cannot be folded into an existing prompt — it must run before Step 9, not within it. The pattern directly mirrors Step 5.5 (prompt classification), which is already proven in the codebase.

• agentic_bug_step6_test_plan_LLM.prompt (MODIFY) — Confirmed. The existing prompt's "Test Location" section (lines 66-69, 106-107) presents (append) and (new) as equally valid options and the example reinforces (new). This is a targeted rewording, not a new module.

• agentic_bug_orchestrator_python.prompt (MODIFY) — Confirmed. The orchestrator prompt needs Step 8.5 added to the step sequence table, timeout dict, and post-step parsing logic. Follows the exact same integration pattern as Step 5.5.

No opportunities for extension-over-creation were missed.

Implementation Approach

1. Step 8.5 (new prompt): Lightweight LLM classification prompt that receives all prior step outputs (1-8) and outputs E2E_NEEDED: yes or E2E_NEEDED: no. If no, the orchestrator skips Step 9 entirely. The prompt will check: "Is there a user-facing behavior not already covered by the unit test from Step 7?"

2. Step 6 (prompt modification): Reword the "Plan test location" section to state that appending to the nearest existing test file is the preferred default. Update the example in the output format to show (append) instead of (new). Only create a new file when no relevant test file exists.

3. Orchestrator (prompt modification):

   • Add Step 8.5 to the step sequence table (between Step 8 and Step 9)
   • Add timeout entry: 8.5: 240.0 (lightweight classification, similar to Step 1)
   • Add post-step parsing: check for E2E_NEEDED: no → skip Step 9 and set a context flag
   • Update step count references (11-step → 12-step)
   • The agentic_bug_python.prompt dependency also needs its step count description updated

Key Design Decisions (resolved)

• Step 8.5 timeout: 240s (matches other lightweight classification steps like Step 1). No complex reasoning needed — just compare unit test coverage against user-facing behavior.
• Skip mechanism: When E2E_NEEDED: no, the orchestrator skips Step 9 entirely (does not run it). This is different from the existing E2E_SKIP which runs Step 9 and lets it self-skip internally. Both mechanisms can coexist for defense-in-depth.
• Output format: E2E_NEEDED: yes/no — simple, parseable, consistent with existing marker patterns (DEFECT_TYPE: code/prompt, E2E_SKIP:, etc.)

Assumptions

• Step 8.5's classification is advisory — if it says E2E_NEEDED: no but the user disagrees, the unit test from Step 7 still provides coverage
• The existing E2E_SKIP mechanism inside Step 9 remains as a fallback (defense-in-depth)
• Step count updates (11→12) are cosmetic and don't affect orchestrator logic (it iterates the steps list, not a counter)

---

Proceeding to Step 8: Analyze Prompt Changes

[prompt-driven-github]

Step 8: Prompt Change Analysis

Status: Changes Analyzed

Summary

Two prompt modifications and one new prompt creation are needed. Step 6's prompt gets rewording to prefer (append) over (new) for test file placement. The orchestrator prompt gets a new Step 8.5 entry and conditional logic. A new lightweight Step 8.5 classification prompt is created following the proven Step 5.5 pattern.

Prompt Modifications

prompts/agentic_bug_step6_test_plan_LLM.prompt

Current state: The "Plan test location" section (task item 4, lines 66-69) and the "Test Location" output template (lines 106-107) present (append) and (new) as equally valid options. The example shows (append) or (new) side by side with no preference.

Changes needed:

1. Update requirement in task item 4 ("Plan test location"): Replace the current neutral guidance with explicit append-first preference:
   • Add: "Prefer appending to the nearest existing test file for the affected module. Only create a new test file when no relevant test file exists in the project."
   • Add: "Search for existing test files matching patterns like test_{module}.py, test_{module}_*.py, or {module}.test.ts before deciding."
2. Update the output template "Test Location" section (line 106-107): Change the example from:

   - **File:** `tests/test_module.py` (append) or `tests/test_module_bug_123.py` (new)
   

   to:

   - **File:** `tests/test_module.py` (append) — preferred: appending to nearest existing test file
   - **Justification:** [Why appending here / or why a new file is necessary if no relevant test file exists]
   

3. Add a constraint in the "Important" section (bottom of prompt): Add bullet: "Default to appending tests to the nearest existing test file for the affected module. Only create a new file when no relevant test file exists. This prevents /tests directory bloat from many small issue-specific files."

prompts/agentic_bug_orchestrator_python.prompt

Current state: Defines a 10-step sequence (Steps 1-10) with Step 5.5 as the only fractional step. Step 9 (E2E test) runs unconditionally after Step 8.

Changes needed:

1. Update "Role & Scope" (line 7): Change "10-step" to "10-step (plus sub-steps 5.5 and 8.5)" or similar to reflect the new sub-step.
2. Add Step 8.5 to the Step Sequence table (lines 30-41): Insert a new row between Step 8 and Step 9:

   | 8.5 | agentic_bug_step8_5_e2e_classification_LLM | Classify whether E2E test is needed |
   

3. Add Step 8.5 to the Context Accumulation section (lines 43-51): Add:

   - Step 8.5 receives: issue_content, step1_output through step8_output
   - Step 9 receives: ... plus step8_5_output (and is only run if E2E_NEEDED: yes)
   

4. Add a new Early Exit Condition for Step 8.5 to the hard-stops table (lines 62-71): Add:

   | 8.5 | E2E not needed | Output contains "E2E_NEEDED: no" |
   

   Note: This is a "soft exit" — it skips Step 9 but continues to Step 10 (PR creation). It is NOT a hard stop that terminates the entire workflow. The orchestrator should skip Step 9 and proceed directly to Step 10 when E2E_NEEDED: no.
5. Add Step 8.5 timeout: Mention that BUG_STEP_TIMEOUTS should include 8.5: 240.0 (lightweight classification, similar to Step 1).
6. Update Requirement 3: Change "Run 10 steps sequentially" to "Run up to 12 steps sequentially (including sub-steps 5.5 and 8.5)" to reflect the actual step count.

prompts/agentic_bug_python.prompt (dependency update)

Current state: Line 7 says "10-step investigation process."

Changes needed:

1. Update description (line 7): Change "10-step" to "multi-step" (more resilient to future step count changes) in the Role & Scope section.

New Prompt

prompts/agentic_bug_step8_5_e2e_classification_LLM.prompt

New prompt to create:

• Role: Lightweight classifier that determines whether an E2E test is warranted based on the bug type and existing unit test coverage from Step 7.
• Requirements:
  1. Analyze the bug's nature: is it purely internal logic (e.g., wrong calculation, off-by-one) or does it have user-facing/integration-level behavior (e.g., CLI output, API response, cross-module data flow)?
  2. Review the unit test from Step 7 output: does it already cover the user-facing behavior, or only internal function behavior?
  3. Classify as E2E_NEEDED: yes if there is user-facing behavior not already covered by the unit test (e.g., CLI command produces wrong output, API endpoint returns wrong status, cross-module integration is broken).
  4. Classify as E2E_NEEDED: no if the bug is purely internal and the unit test fully covers the observable behavior (e.g., a helper function returns wrong value, already tested directly).
  5. Default to E2E_NEEDED: yes when uncertain — it's better to generate an unnecessary E2E test than to miss one.
  6. Output must contain exactly one line matching E2E_NEEDED: yes or E2E_NEEDED: no for machine parsing.
  7. Provide brief justification (2-3 sentences) explaining the classification decision.
• Dependencies:
  • Receives same context as other steps: issue_content, step1_output through step8_output
  • No <include> dependencies needed — this is a pure classification prompt with no code generation
• Structure: Follow the Step 5.5 prompt pattern (classification with machine-readable output marker). Include:
  • Context section with step number ("step 8.5 of 12")
  • Inputs section (issue_url, repo_owner, repo_name, issue_number)
  • Previous Steps Output (step1 through step8)
  • Classification criteria (internal-only vs. user-facing behavior)
  • Output template with E2E_NEEDED: marker
  • GitHub comment posting instruction

Extension Analysis (REQUIRED):

• Modules considered:
  • agentic_bug_step9_e2e_test_LLM.prompt — Already has an E2E_SKIP escape hatch at the end of its execution, but this runs AFTER the expensive 2000s timeout step has already started. The whole point of the gate is to avoid starting Step 9 at all.
  • agentic_bug_step8_verify_LLM.prompt — Could theoretically add E2E classification to the verify step, but this would overload Step 8 with two distinct responsibilities (verify unit test correctness AND classify E2E need), violating the one-prompt-one-purpose principle.
• Why not extendable: The Step 9 E2E_SKIP mechanism cannot prevent the 2000s timeout cost because it only triggers after Step 9's agent is already running. Step 8 has a different purpose (test verification) and adding classification would conflate concerns.
• Capability gap: A pre-Step-9 gate that makes a quick yes/no decision (240s timeout) to avoid the expensive Step 9 invocation (2000s timeout) for simple internal bugs. This follows the exact same pattern as Step 5.5 (pre-Step-6 classification gate).

Interface Changes

• No breaking changes. All modifications are additive:
  • Step 8.5 is a new sub-step that slots between existing Steps 8 and 9
  • Step 9 behavior is unchanged — it just may be skipped when Step 8.5 says E2E_NEEDED: no
  • Step 6's output format gains a Justification field but remains backward-compatible
  • Step 10 (PR) is unaffected — it still runs regardless of whether Step 9 was skipped
• Orchestrator skip logic: When E2E_NEEDED: no, the orchestrator skips Step 9 and proceeds to Step 10. The step9_output context variable should be set to a placeholder like "Step 9 skipped: E2E test not needed (classified at Step 8.5)" so Step 10's template doesn't break on a missing variable.

Risk Assessment

• Low: Both changes are well-precedented:
  • Step 8.5 follows the exact pattern of Step 5.5 (already proven in the codebase)
  • Step 6 rewording is a prompt text change with no structural impact
  • The E2E_NEEDED: no default-to-yes policy ensures no E2E coverage is lost when classification is uncertain
• Mitigation: The "default to yes" requirement (Requirement 5 of the new prompt) ensures that ambiguous cases still get E2E tests, so the worst case is the same as today's behavior (always running Step 9).

---

Proceeding to Step 9: Implement Changes

[prompt-driven-github]

Step 9: Implement Changes

Status: Changes Applied

Files Modified

FILES_MODIFIED: prompts/agentic_bug_step6_test_plan_LLM.prompt, prompts/agentic_bug_orchestrator_python.prompt, prompts/agentic_bug_python.prompt, README.md

Files Created

FILES_CREATED: prompts/agentic_bug_step8_5_e2e_classification_LLM.prompt

Summary of Changes

prompts/agentic_bug_step6_test_plan_LLM.prompt

• Rewrote "Plan test location" section to make append to existing test file the explicit default
• Added guidance: only create new test files when no relevant existing test file exists
• Updated output template example to show (append — preferred default) with justification field
• Prevents /tests directory bloat from many small issue-specific files

prompts/agentic_bug_orchestrator_python.prompt

• Changed "10-step" to "multi-step" in Role & Scope description
• Updated requirement 3 to list all steps including 8.5 in the sequence
• Added requirement 9: Step 8.5 parsing logic for E2E_NEEDED: yes/no with skip-Step-9 behavior
• Added Step 8.5 row to Step Sequence table
• Updated Context Accumulation section with Step 8.5 data flow
• Added Step 8.5 to Early Exit Conditions table (skip Step 9 when E2E_NEEDED: no)

prompts/agentic_bug_python.prompt

• Changed "10-step investigation process" to "multi-step investigation process" in Role & Scope

prompts/agentic_bug_step8_5_e2e_classification_LLM.prompt (new)

• Lightweight classification prompt following the Step 5.5 pattern
• Reviews bug scope, unit test coverage, and E2E gap
• Outputs E2E_NEEDED: yes or E2E_NEEDED: no
• Defaults to yes when uncertain (safe default)
• Avoids the expensive 2000s Step 9 timeout for simple internal bugs

README.md

• Updated Step 6 description to mention append-first preference
• Added Step 8.5 description (E2E classification gate)
• Updated Step 9 description to note it only runs when Step 8.5 determines E2E is needed

SCOPE_VIOLATION: No

Worktree Location

Changes are in: /tmp/pdd_job_f5zKQsQTPhqFz7D1qcLQ_66zvcuey/.pdd/worktrees/change-issue-642

Next Steps

After review, run pdd sync on modified prompts to regenerate code.

---

Proceeding to Step 10: Identify Issues

[prompt-driven-github]

Step 10: Architecture Update

Status: Tags Added (no architecture.json)

Tags Added/Updated

Prompt	Action	Interface Type
prompts/agentic_bug_step8_5_e2e_classification_LLM.prompt	Already tagged (reason)	N/A (LLM prompt)
prompts/agentic_bug_step6_test_plan_LLM.prompt	Already tagged (reason)	N/A (LLM prompt)
prompts/agentic_bug_orchestrator_python.prompt	Added tags (reason, interface)	module
prompts/agentic_bug_python.prompt	Added tags (reason, interface, dependency)	module

ARCHITECTURE_FILES_MODIFIED: prompts/agentic_bug_orchestrator_python.prompt, prompts/agentic_bug_python.prompt

---

Proceeding to Step 11: Identify Issues