Message: Context missing key for step 5: '"url"' when doing pdd change

[DFectuoso]

dfect@Santiagos-MacBook-Pro pdd-cal % pdd change https://github.com/DFectuoso/pdd-cal/issues/3
2026-01-31 15:21:13,686 - pdd.llm_invoke - INFO - Using user-specific LLM model CSV: /Users/dfect/.pdd/llm_model.csv
2026-01-31 15:21:13,699 - pdd.llm_invoke - INFO - LiteLLM disk cache configured at /Users/dfect/code/pdd-cal/litellm_cache.sqlite
Checking for updates...
Fetching issue #3 from DFectuoso/pdd-cal...
Using current directory as repository: /Users/dfect/code/pdd-cal
Starting Agentic Change Orchestrator...
Implementing change for issue #3: "event calendar"
[Step 1/13] Search for duplicate issues...
Successfully loaded prompt: agentic_change_step1_duplicate_LLM
-> Issue #3 is now the canonical issue for this feature request and can proceed ...
[Step 2/13] Check if already implemented...
Successfully loaded prompt: agentic_change_step2_docs_LLM
-> Status: Not Implemented - The workflow should continue to Step 3: Identif...
[Step 3/13] Research to clarify specifications...
Successfully loaded prompt: agentic_change_step3_research_LLM
-> The research comment has been posted to GitHub issue #3 and the workflow is r...
[Step 4/13] Verify requirements are clear...
Successfully loaded prompt: agentic_change_step4_clarify_LLM
-> The workflow can proceed to Step 5: Documentation Changes.
[Step 5/13] Analyze documentation changes needed...
Successfully loaded prompt: agentic_change_step5_docs_change_LLM
Status: Failed
Message: Context missing key for step 5: '"url"'
Cost: $4.5006
Model: anthropic

--- Command Execution Summary ---
Step 1 (change): Cost: $4.500635, Model: anthropic
Total Estimated Cost: $4.500635

📦 Debug snapshot saved to /Users/dfect/code/pdd-cal/.pdd/core_dumps/pdd-core-20260131T232730Z.json (attach when reporting bugs)

[gltanaka]

Step 1: Duplicate Check

Status: Duplicate of #392 (which is itself a duplicate of #319)

Search Performed

• Searched for: "Context missing key", "step 5", "url missing", "pdd change fails"
• Issues reviewed: 12

Findings

This issue is a duplicate of existing issues tracking the same bug:

Issue	Title	Error Message	Status
#392	pdd change fails at Step 5 with 'Context missing key for step 5: url'	'"url"'	OPEN
#319	pdd change: failure at Step 5	'\n "type"'	OPEN
#376	Bug: pdd change fails at step 5 due to template injection in step outputs	'\n "type"'	OPEN

Root Cause (from #376):
The Agentic Change Orchestrator uses Python's str.format() to substitute variables into prompt templates. When step outputs contain curly braces (e.g., JSON content like {"url": "..."} or {"type": "..."}), Python's .format() method misinterprets them as template placeholders, causing a KeyError.

This issue vs #392:
Both issues have the identical error message: Context missing key for step 5: '"url"'

The difference between "url" and "type" across issues is simply which JSON field name happens to appear first in the content that triggers the bug.

Fix Status:

• pdd change: failure at Step 5 #319 has extensive investigation and shows "Bug fix complete!" with all 27 e2e tests passing
• However, the issue remains OPEN and users continue experiencing this bug (as evidenced by pdd change fails at Step 5 with 'Context missing key for step 5: url' #392 and now Message: Context missing key for step 5: '"url"' when doing pdd change #448)
• This suggests either the fix hasn't been released, or is incomplete

Recommendation:
Close this issue as a duplicate of #392 and consolidate tracking in #319/392. The unique information from this report (DFectuoso's reproduction case with pdd-cal repo) should be preserved as additional evidence.

---

Issue #448 identified as duplicate of #392. Closing with reference.

[gltanaka]

Closing as duplicate of #392 (which is itself tracking the same root cause as #319 and #376). See the duplicate check comment above for details.

[gltanaka]

Step 2: Documentation Check

Status: Confirmed Bug

Documentation Reviewed

• CONTRIBUTING.md - PDD workflow and contribution guidelines
• CLAUDE.md - Development guidelines
• Issue Bug: pdd change fails at step 5 due to template injection in step outputs #376 - Detailed root cause analysis
• Issue pdd change: failure at Step 5 #319 - Canonical issue for this bug
• Issue pdd change fails at Step 5 with 'Context missing key for step 5: url' #392 - Identical error message duplicate
• pdd/agentic_change_orchestrator.py:597-600 - Bug location

Findings

This is a confirmed bug, not user error or a feature request. The documentation does not describe this behavior as expected or intentional.

Root Cause (from #376):
Python's str.format() method in the Agentic Change Orchestrator misinterprets JSON curly braces (e.g., {"url": "..."}) in step outputs as template placeholders, causing KeyError exceptions. This is a clear code defect.

Why this is NOT user error:

• @DFectuoso followed the documented usage: pdd change <github-issue-url>
• The command executed correctly through Steps 1-4
• The failure occurred due to internal template processing, not user input

Why this is NOT a feature request:

• The documented purpose of pdd change is to implement changes for GitHub issues
• The command should complete successfully when given a valid issue URL
• Failing at Step 5 with a template key error contradicts the expected behavior

Code defect location:

# pdd/agentic_change_orchestrator.py:597-600
try:
    formatted_prompt = prompt_template.format(**context)
except KeyError as e:
    return False, f"Context missing key for step {step_num}: {e}", ...

Note: This issue was already closed as a duplicate of #392 in Step 1. The canonical issue for tracking is #319 with detailed analysis in #376.

---

Proceeding to Step 3: Triage (though issue is already closed as duplicate)

[gltanaka]

Step 3: Triage

Status: Proceed

Summary

We have sufficient information to reproduce this bug:

Expected behavior: The pdd change command should complete all 13 steps successfully.

Actual behavior: The command fails at Step 5 with:

Message: Context missing key for step 5: '"url"'

Root cause identified: Python's str.format() in agentic_change_orchestrator.py:598 misinterprets JSON curly braces (e.g., {"url": "..."}) in prior step outputs as template placeholders, causing a KeyError.

Reproduction path:

1. Run pdd change <github_issue_url>
2. Steps 1-4 succeed, producing outputs that contain JSON with curly braces
3. Step 5 fails when prompt_template.format(**context) tries to interpolate {url} from the JSON

Code location: pdd/agentic_change_orchestrator.py:598

Related issues: #319 (canonical), #376 (root cause analysis), #392 (duplicate)

---

Proceeding to Step 4: Reproduction

[gltanaka]

Step 4: Reproduction

Status: Reproduced

Environment

• OS: macOS Darwin 25.2.0
• Python: 3.11+
• pdd version: Current HEAD (commit a6a33b4e)

Root Cause Identified

The bug occurs because agentic_change_orchestrator.py is missing a call to preprocess() before calling .format() on prompt templates.

Comparison of orchestrators:

Orchestrator	Line	Has preprocess() call?
agentic_architecture_orchestrator.py	299-302	✅ Yes - preprocess(..., double_curly_brackets=True)
agentic_change_orchestrator.py	593-598	❌ No - directly calls .format()

Steps Taken

1. Located the error in agentic_change_orchestrator.py:598:

   formatted_prompt = prompt_template.format(**context)

2. Compared with agentic_architecture_orchestrator.py:299-305:

   # Preprocess to expand <include> tags and escape curly braces
   exclude_keys = list(context.keys())
   prompt_template = preprocess(prompt_template, recursive=True, double_curly_brackets=True, exclude_keys=exclude_keys)
   formatted_prompt = prompt_template.format(**context)

3. Reproduced with minimal test:

   from pdd.preprocess import preprocess
   
   # Template with JSON (simulates included file content)
   template = 'Data: {"url": "test"}, Step: {step4_output}'
   context = {'step4_output': 'output'}
   
   # WITHOUT preprocess (bug)
   template.format(**context)  # KeyError: '"url"'
   
   # WITH preprocess (fix)
   preprocessed = preprocess(template, double_curly_brackets=True, exclude_keys=['step4_output'])
   preprocessed.format(**context)  # Success!

Observed Behavior

KeyError: '"url"'

This error occurs because Python's .format() interprets {"url": ...} in the template as a format placeholder looking for a key named "url" (with quotes).

Affected Files

• pdd/agentic_change_orchestrator.py:598 - Missing preprocess() call before .format()

Fix Required

Add the same preprocessing logic from agentic_architecture_orchestrator.py to agentic_change_orchestrator.py:

# Before line 597, add:
from pdd.preprocess import preprocess

# Replace lines 597-598 with:
exclude_keys = list(context.keys())
prompt_template = preprocess(prompt_template, recursive=True, double_curly_brackets=True, exclude_keys=exclude_keys)
formatted_prompt = prompt_template.format(**context)

Notes

• This bug affects any pdd change run where step outputs or included files contain JSON or code with curly braces
• The same pattern should be applied to lines 713, 727, and 790 in agentic_change_orchestrator.py (steps 11, 12, and 13)
• Related canonical issues: pdd change: failure at Step 5 #319, Bug: pdd change fails at step 5 due to template injection in step outputs #376, pdd change fails at Step 5 with 'Context missing key for step 5: url' #392

---

Proceeding to Step 5: Root Cause Analysis

[gltanaka]

Step 5: Root Cause Analysis

Root Cause Identified: Yes

Summary

The bug occurs when Python's str.format() method is called on prompt templates that contain included files with JSON or dictionary-like content. The <include>docs/prompting_guide.md</include> directive in agentic_change_step5_docs_change_LLM.prompt brings in content with unescaped curly braces (e.g., {"url": "..."}). When .format(**context) runs at line 598, Python interprets these as template placeholders, causing KeyError: '"url"' because "url" is not a key in the context dictionary.

Technical Details

• Location: pdd/agentic_change_orchestrator.py:598
• Problematic Code:

try:
    formatted_prompt = prompt_template.format(**context)
except KeyError as e:
    return False, f"Context missing key for step {step_num}: {e}", total_cost, model_used, []

• Why it fails:
  1. Step 5's prompt template includes docs/prompting_guide.md via <include> tags
  2. prompting_guide.md contains JSON examples like {"url": "https://fastapi.tiangolo.com/..."} (line 231)
  3. load_prompt_template() expands includes but does NOT escape curly braces
  4. When .format(**context) executes, Python sees {"url": ...} and tries to substitute a key named "url"
  5. Since "url" is not in the context dict, Python raises KeyError: '"url"'

External Research

• Search terms: "Python str.format KeyError curly braces JSON template injection"
• Sources checked: Python docs, GitHub issues (Do not use Python's str.format() for expanding placeholders in commands pypa/cibuildwheel#840, loguru#1008, llama_index#14310)
• Relevant findings:
  • This is a well-known Python issue - using str.format() on strings containing literal braces requires escaping them as {{ and }}
  • Multiple libraries (loguru, llama_index, tavern) have encountered identical issues
  • Standard fix: preprocess strings to double curly braces before calling .format()
  • Reference: Python String Formatting with Curly Braces

Repository History

• Relevant commits:

  • 9c2e7696 (2026-01-28) "revert: Remove template preprocessing from agentic orchestrators" - This commit explicitly removed the preprocess() call that would have prevented this bug
  • d04d16d1 (2026-01-21) - Added the try/except around .format() but without preprocessing

• Related PRs: The revert commit mentions "PRs hello example in repo is confusing #48-51 that were causing agentic verify to fail for non-Python modules"

• Similar past issues:

  • pdd change: failure at Step 5 #319 (OPEN): Canonical issue - "pdd change: failure at Step 5" with KeyError '\n "type"'
  • Bug: pdd change fails at step 5 due to template injection in step outputs #376 (OPEN): Detailed root cause analysis with fix recommendations
  • pdd change fails at step 5 when previous step output contains JSON with curly braces #352 (CLOSED): "pdd change fails at step 5 when previous step output contains JSON"
  • pdd change: KeyError '\n "type"' in Step 5 prompt template due to unescaped JSON in included prompting_guide.md #373 (CLOSED): "pdd change: KeyError '\n "type"' in Step 5 prompt template"
  • pdd change fails at Step 5 with 'Context missing key for step 5: url' #392 (OPEN): Duplicate of pdd change: failure at Step 5 #319 with identical error

Experiments Performed

1. Examined prompting_guide.md: Confirmed it contains JSON with curly braces at lines 231, 262, 291, 319, 331, 343, 355 (e.g., {"url": "..."})

2. Compared with working code: agentic_architecture_orchestrator.py:299-302 correctly uses:

   exclude_keys = list(context.keys())
   prompt_template = preprocess(prompt_template, recursive=True, double_curly_brackets=True, exclude_keys=exclude_keys)

   This escapes curly braces BEFORE calling .format(), while preserving template placeholders.

3. Verified revert commit: The 9c2e7696 commit removed preprocessing from agentic_change_orchestrator.py to fix a different issue (verify failures for non-Python modules), but this introduced the current bug.

Fix Direction

The fix must add preprocess(..., double_curly_brackets=True) before calling .format() at four locations in agentic_change_orchestrator.py:

1. Line 598 (main step loop)
2. Line 713 (step 11 - identify issues)
3. Line 727 (step 12 - fix issues)
4. Line 790 (step 13 - create PR)

The pattern from agentic_architecture_orchestrator.py should be replicated:

from pdd.preprocess import preprocess

# Before .format():
exclude_keys = list(context.keys())
prompt_template = preprocess(prompt_template, recursive=True, double_curly_brackets=True, exclude_keys=exclude_keys)
formatted_prompt = prompt_template.format(**context)

The exclude_keys parameter ensures that actual template placeholders like {issue_url} remain as single braces (so .format() can substitute them), while other curly braces from included content are escaped.

Note: The original revert was done to fix verify failures for non-Python modules. Any fix must be tested against that scenario to avoid regression.

---

Proceeding to Step 5.5: Prompt Classification

[gltanaka]

Step 5.5: Prompt Classification

Classification: Code Bug

DEFECT_TYPE: code

Analysis

This is a code bug, not a prompt defect. The prompt specification states that templates should be formatted with context variables to produce working prompts. The code fails to achieve this because it doesn't escape literal curly braces from included files before calling Python's .format().

Evidence

• Prompt specifies: "Format template with: issue_url, repo_owner, repo_name, issue_number, issue_content, issue_author, step1_output, step2_output, etc." (line 52 of prompts/agentic_change_orchestrator_python.prompt)

• Code implements: Directly calls prompt_template.format(**context) at line 598 without preprocessing to escape literal braces from included files

• User expects: Template formatting should work correctly when step prompts include files containing JSON examples like {"url": "https://..."}

Comparison with Working Code

The agentic_architecture_orchestrator.py correctly handles this at lines 299-302:

# Preprocess to expand <include> tags and escape curly braces
exclude_keys = list(context.keys())
prompt_template = preprocess(prompt_template, recursive=True, double_curly_brackets=True, exclude_keys=exclude_keys)

The agentic_change_orchestrator.py is missing this preprocessing step, which was deliberately removed by revert commit 9c2e7696 on 2026-01-28.

Conclusion

The code does not correctly implement the prompt specification's intent. Both orchestrator prompts have identical specifications for template formatting - one implementation (architecture) handles curly brace escaping, the other (change) doesn't. This is a code inconsistency, not a prompt defect.

The fix is to add the same preprocess(..., double_curly_brackets=True) call at the four identified locations in agentic_change_orchestrator.py.

Proceeding with test generation.

---

Proceeding to Step 6: Test Plan

[gltanaka]

Step 6: Test Plan

Existing Test Coverage

• Test file: tests/test_agentic_change_orchestrator.py
• Current coverage: The existing tests use mocked load_prompt_template that returns a mock template with .format() method. This completely bypasses the actual prompt loading and curly bracket escaping logic.
• Gap: No tests verify that the orchestrator correctly preprocesses templates containing JSON (with literal curly braces) before calling .format(). The existing tests mock load_prompt_template to return a simple MagicMock, so the actual template content with <include> tags and JSON examples is never processed.

Root Cause Recap

The bug occurs at agentic_change_orchestrator.py:598 when:

1. Step 5 loads prompt template agentic_change_step5_docs_change_LLM.prompt
2. The template includes docs/prompting_guide.md via <include> tag
3. prompting_guide.md contains JSON examples like {"url": "https://..."}
4. When .format(**context) is called, Python interprets {"url": as a template placeholder
5. This raises KeyError: '"url"'

Working code comparison: agentic_architecture_orchestrator.py:299-302 correctly uses:

exclude_keys = list(context.keys())
prompt_template = preprocess(prompt_template, recursive=True, double_curly_brackets=True, exclude_keys=exclude_keys)

Proposed Tests

Test 1: Template with JSON content formats without KeyError (Main Bug)

• Input: Load real agentic_change_step5_docs_change_LLM.prompt template (which includes docs/prompting_guide.md containing JSON), apply preprocessing with double_curly_brackets=True, then call .format(**context)
• Expected: No KeyError, JSON curly braces are escaped to {{}} and rendered as literal {} after formatting
• Actual (before fix): KeyError: '"url"' at line 598

Test 2: Step 5 orchestrator integration test

• Input: Run orchestrator with mocked LLM but REAL template loading and formatting (don't mock load_prompt_template)
• Expected: Step 5 executes without KeyError, proceeds to next step
• Actual (before fix): Fails with Context missing key for step 5: '"url"'

Test 3: Step 11/12/13 templates with JSON content (Same pattern at other locations)

• Input: Load templates for steps 11, 12, 13 which also lack preprocessing
• Expected: Templates with included JSON format correctly
• Actual (before fix): Potential KeyError if templates include files with JSON

Test 4: Regression test - verify preprocess+format pattern works

• Input: Template string containing {"key": "value"} + context with different keys
• Expected: After preprocess(..., double_curly_brackets=True, exclude_keys=[context_keys]), .format(**context) succeeds
• Pattern: This is the same pattern used in agentic_architecture_orchestrator.py

Test Location

• File: tests/test_e2e_issue_448_step5_keyerror.py (new file, following pattern from test_e2e_issue_357_step9_keyerror.py)
• Framework: pytest
• Naming convention: Matches existing e2e issue-specific test files

Notes

1. Similar existing tests: tests/test_e2e_issue_357_step9_keyerror.py tests the same class of bug (KeyError from unescaped curly braces) in a different orchestrator. This new test file will follow the same pattern.

2. Test design considerations:

   • Tests should use REAL load_prompt_template to load actual prompts (don't mock the template loading)
   • Tests should use REAL preprocess function to verify escaping works
   • Mock only the LLM calls (run_agentic_task) to avoid actual API costs
   • Tests should verify both the direct template formatting AND the full orchestrator code path

3. Affected code locations requiring the fix:

   • Line 598 (main step loop) - Primary bug location
   • Line 713 (step 11)
   • Line 727 (step 12)
   • Line 790 (step 13)

4. Required fixture: Mock working directory with .git to simulate a git repo (same as test_e2e_issue_357_step9_keyerror.py)

---

Proceeding to Step 7: Generate Test

[gltanaka]

Step 7: Generated Test

Test File

tests/test_e2e_issue_448_step5_keyerror.py

Test Code

"""
E2E Test for Issue #448: KeyError '"url"' in step 5 prompt template due to unescaped JSON braces

This E2E test exercises the prompt loading and formatting code path for Step 5 of the
agentic change workflow. It verifies that the orchestrator correctly handles templates
that include files containing JSON content.

The bug: The agentic_change_orchestrator.py does NOT call preprocess() before format().
When a template includes docs/prompting_guide.md (which contains JSON like {"url": "..."}),
and includes ARE expanded, the JSON curly braces cause KeyError during format().

The fix requires calling preprocess(..., double_curly_brackets=True) before format(),
as done correctly in agentic_architecture_orchestrator.py:299-302.
"""

import pytest
import inspect

class TestIssue448Step5KeyErrorE2E:
    def test_step5_template_without_preprocessing_causes_keyerror(self, mock_cwd):
        """BUG REPRODUCTION: Expanding includes WITHOUT double_curly causes KeyError."""
        from pdd.load_prompt_template import load_prompt_template
        from pdd.preprocess import preprocess

        context = {
            "issue_url": "https://github.com/test/repo/issues/448",
            "repo_owner": "test", "repo_name": "repo",
            "issue_number": 448, "issue_content": "Test issue",
            "step1_output": "...", "step2_output": "...",
            "step3_output": "...", "step4_output": "...",
        }

        template = load_prompt_template("agentic_change_step5_docs_change_LLM")
        processed = preprocess(template, recursive=True, double_curly_brackets=False)

        # THE BUG: format() raises KeyError on JSON braces
        with pytest.raises(KeyError) as exc_info:
            processed.format(**context)
        assert '"url"' in str(exc_info.value)

    def test_change_orchestrator_missing_preprocess_call(self):
        """CODE INSPECTION: Verify change orchestrator is missing preprocess call."""
        import pdd.agentic_change_orchestrator as change_orch
        source = inspect.getsource(change_orch)

        has_preprocess = "from pdd.preprocess import preprocess" in source
        has_double_curly = "double_curly_brackets=True" in source

        if not (has_preprocess and has_double_curly):
            pytest.fail("BUG: agentic_change_orchestrator.py is missing preprocess()")

What This Test Verifies

The tests verify three aspects of the bug:

Test	What It Checks	Before Fix	After Fix
test_step5_template_without_preprocessing_causes_keyerror	JSON braces cause KeyError when not escaped	PASS (KeyError raised as expected)	PASS
test_step5_template_with_correct_preprocessing_works	Correct preprocessing prevents KeyError	PASS	PASS
test_change_orchestrator_missing_preprocess_call	Code inspection for bug presence	FAIL (bug detected)	PASS
test_change_orchestrator_should_match_architecture_pattern	Compare to working orchestrator	FAIL (pattern missing)	PASS
test_all_format_calls_should_be_preceded_by_preprocess	All 4 format() calls need fix	FAIL (0/4 have preprocess)	PASS

Test Results (Current Buggy Code)

============================= test session starts ==============================
collected 9 items

test_e2e_issue_448_step5_keyerror.py::TestIssue448Step5KeyErrorE2E::test_step5_template_without_preprocessing_causes_keyerror PASSED
test_e2e_issue_448_step5_keyerror.py::TestIssue448Step5KeyErrorE2E::test_step5_template_with_correct_preprocessing_works PASSED
test_e2e_issue_448_step5_keyerror.py::TestIssue448Step5KeyErrorE2E::test_change_orchestrator_missing_preprocess_call FAILED
test_e2e_issue_448_step5_keyerror.py::TestIssue448CompareOrchestrators::test_architecture_orchestrator_has_correct_pattern PASSED
test_e2e_issue_448_step5_keyerror.py::TestIssue448CompareOrchestrators::test_change_orchestrator_should_match_architecture_pattern FAILED
test_e2e_issue_448_step5_keyerror.py::TestIssue448RegressionPrevention::test_prompting_guide_contains_json_braces PASSED
test_e2e_issue_448_step5_keyerror.py::TestIssue448RegressionPrevention::test_step5_template_includes_prompting_guide PASSED
test_e2e_issue_448_step5_keyerror.py::TestIssue448RegressionPrevention::test_preprocess_expands_includes PASSED
test_e2e_issue_448_step5_keyerror.py::TestIssue448OtherAffectedLocations::test_all_format_calls_should_be_preceded_by_preprocess FAILED

=========================== short test summary info ============================
FAILED: BUG DETECTED - agentic_change_orchestrator.py missing preprocess()
FAILED: Change orchestrator doesn't match architecture orchestrator pattern
FAILED: 4 format() calls found but no preprocess(..., double_curly_brackets=True)
========================= 3 failed, 6 passed in 2.93s ==========================

Running the Test

pytest -vv tests/test_e2e_issue_448_step5_keyerror.py

Fix Required

Add the preprocessing pattern from agentic_architecture_orchestrator.py:299-302 to agentic_change_orchestrator.py:

# At top of file, add import:
from pdd.preprocess import preprocess

# Before each format() call (lines 598, 713, 727, 790), add:
exclude_keys = list(context.keys())
prompt_template = preprocess(prompt_template, recursive=True, double_curly_brackets=True, exclude_keys=exclude_keys)

---

Proceeding to Step 8: Verification

[gltanaka]

Step 8: Verification Complete

Test Execution

============================= test session starts ==============================
platform darwin -- Python 3.12.11, pytest-8.3.5, pluggy-1.6.0
collecting ... collected 9 items

tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448Step5KeyErrorE2E::test_step5_template_without_preprocessing_causes_keyerror PASSED [ 11%]
tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448Step5KeyErrorE2E::test_step5_template_with_correct_preprocessing_works PASSED [ 22%]
tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448Step5KeyErrorE2E::test_change_orchestrator_missing_preprocess_call FAILED [ 33%]
tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448CompareOrchestrators::test_architecture_orchestrator_has_correct_pattern PASSED [ 44%]
tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448CompareOrchestrators::test_change_orchestrator_should_match_architecture_pattern FAILED [ 55%]
tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448RegressionPrevention::test_prompting_guide_contains_json_braces PASSED [ 66%]
tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448RegressionPrevention::test_step5_template_includes_prompting_guide PASSED [ 77%]
tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448RegressionPrevention::test_preprocess_expands_includes PASSED [ 88%]
tests/test_e2e_issue_448_step5_keyerror.py::TestIssue448OtherAffectedLocations::test_all_format_calls_should_be_preceded_by_preprocess FAILED [100%]

=================================== FAILURES ===================================
TestIssue448Step5KeyErrorE2E.test_change_orchestrator_missing_preprocess_call:
  Failed: BUG DETECTED (Issue #448): agentic_change_orchestrator.py is missing 
  the preprocess() call with double_curly_brackets=True.
  
  Current state:
    - preprocess imported: False
    - double_curly_brackets=True used: False

TestIssue448CompareOrchestrators.test_change_orchestrator_should_match_architecture_pattern:
  Failed: BUG (Issue #448): agentic_change_orchestrator.py does NOT use the same 
  preprocessing pattern as agentic_architecture_orchestrator.py.

TestIssue448OtherAffectedLocations.test_all_format_calls_should_be_preceded_by_preprocess:
  Failed: BUG (Issue #448): Found 4 format() calls but no 
  preprocess(..., double_curly_brackets=True) call.
  
  All template format() calls need preprocessing:
    - Line 598 (main step loop)
    - Line 713 (step 11)
    - Line 727 (step 12)
    - Line 790 (step 13)

========================= 3 failed, 6 passed in 1.26s ==========================

Verification Status

PASS: Test correctly detects the bug

The test suite successfully identifies the bug:

• 3 tests FAIL to detect the missing preprocess() call in agentic_change_orchestrator.py
• 6 tests PASS to verify the correct pattern exists in agentic_architecture_orchestrator.py and confirm the conditions that trigger the bug

Summary

Step	Result
Duplicate Check	Completed - #448 linked to canonical #319/#392
Documentation	Completed - Confirmed bug, not user error
Triage	Completed - Proceeded to reproduction
Reproduction	Completed - Bug reproduced
Root Cause	Identified - Missing preprocess() call
Prompt Classification	Completed - Code bug (not prompt defect)
Test Plan	Created - 9 tests designed
Test Generation	Completed - Test file created
Verification	PASS - Tests correctly detect the bug

Bug Details

• Location: pdd/agentic_change_orchestrator.py:598 (and lines 713, 727, 790)
• Root Cause: prompt_template.format(**context) is called without first calling preprocess(..., double_curly_brackets=True), causing JSON curly braces from included files to be misinterpreted as Python format placeholders
• Test File: tests/test_e2e_issue_448_step5_keyerror.py

Next Steps

1. Fix the bug by adding preprocess() call at all 4 locations in agentic_change_orchestrator.py
2. Run the test to confirm the fix (all 9 tests should pass)
3. Run full test suite to check for regressions
4. Submit PR with fix and test

Fix Pattern (from agentic_architecture_orchestrator.py:299-302)

from pdd.preprocess import preprocess

# Before calling format():
exclude_keys = list(context.keys())
prompt_template = preprocess(
    prompt_template, 
    recursive=True, 
    double_curly_brackets=True, 
    exclude_keys=exclude_keys
)
formatted_prompt = prompt_template.format(**context)

---

Proceeding to Step 9: E2E Tests

[gltanaka]

Step 9: E2E Test

Environment Discovery

• Repository language: Python
• Test framework: pytest 8.3.5
• Existing E2E tests found: Yes - tests/test_e2e_issue_357_step9_keyerror.py, tests/test_e2e_issue_419_cli_unpushed_commits.py, and 14 other E2E test files
• Test configuration: pytest.ini, pyproject.toml
• Fixtures/helpers used: mock_cwd, runner (CliRunner), conftest fixtures for PDD_PATH preservation
• Environment variables required: PDD_FORCE_LOCAL=1 for testing without real LLM calls
• Auth/credentials: Mocked via patches - no real API calls needed

E2E Test File

tests/test_e2e_issue_448_change_orchestrator.py

Test Type

Integration (orchestrator workflow with mocked LLM)

What This Test Verifies

The E2E test verifies the template preprocessing and formatting code path that affects the pdd change command. It exercises:

1. Bug scenario demonstration - Shows that preprocessing includes WITHOUT double_curly_brackets=True causes KeyError when JSON content (like {"url": ...}) is present in included files
2. Fix verification - Confirms that preprocessing with double_curly_brackets=True prevents the KeyError
3. Code inspection - Verifies the change orchestrator is missing the preprocessing pattern
4. Orchestrator comparison - Confirms the architecture orchestrator has the correct pattern but the change orchestrator doesn't

Test Execution Result

============================= test session starts ==============================
platform darwin -- Python 3.12.11, pytest-8.3.5
collected 6 items

tests/test_e2e_issue_448_change_orchestrator.py::test_preprocessing_without_double_curly_causes_keyerror PASSED
tests/test_e2e_issue_448_change_orchestrator.py::test_preprocessing_with_double_curly_prevents_keyerror PASSED
tests/test_e2e_issue_448_change_orchestrator.py::test_change_orchestrator_needs_preprocessing_pattern FAILED
tests/test_e2e_issue_448_change_orchestrator.py::test_change_orchestrator_workflow_with_mock_llm PASSED
tests/test_e2e_issue_448_change_orchestrator.py::test_architecture_orchestrator_has_correct_preprocessing PASSED
tests/test_e2e_issue_448_change_orchestrator.py::test_both_orchestrators_should_use_same_pattern FAILED

FAILURES:
test_change_orchestrator_needs_preprocessing_pattern - BUG DETECTED: Missing preprocess() call
test_both_orchestrators_should_use_same_pattern - BUG: Pattern mismatch between orchestrators

=========================== 2 failed, 4 passed ===========================

E2E Test Status

PASS: E2E test correctly detects the bug

The tests correctly identify that:

1. agentic_change_orchestrator.py does NOT import preprocess
2. agentic_change_orchestrator.py does NOT use double_curly_brackets=True
3. The architecture orchestrator correctly uses this pattern at lines 299-302

Combined Test Results (Both Files)

Running both test_e2e_issue_448_step5_keyerror.py (unit tests) and test_e2e_issue_448_change_orchestrator.py (E2E tests):

• 5 tests FAIL (correctly detecting the bug)
• 10 tests PASS (verifying preconditions and fix pattern)

---

Proceeding to Step 10: Create Draft PR

[gltanaka]

Step 10: Draft PR Created

Pull Request

PR #418: Add failing tests for #448

Branch

fix/issue-448

What's Included

• Failing unit test at tests/test_e2e_issue_448_step5_keyerror.py
• Failing E2E test at tests/test_e2e_issue_448_change_orchestrator.py
• Commits: 1

Test Results Summary

Unit tests (9 tests, 3 fail):

• ✅ Template loading works correctly
• ✅ Fix pattern with double_curly_brackets=True prevents KeyError
• ✅ Architecture orchestrator has correct pattern
• ❌ Change orchestrator missing preprocess import
• ❌ Change orchestrator doesn't match architecture pattern
• ❌ 4 format() calls lack preprocessing (lines 598, 713, 727, 790)

E2E tests (6 tests, 2 fail):

• ✅ Demonstrates bug behavior
• ✅ Verifies fix works
• ✅ Integration test passes with mocked LLM
• ❌ Change orchestrator needs preprocessing pattern
• ❌ Both orchestrators should use same pattern

Next Steps for Maintainers

1. Review the failing tests to understand the expected behavior
2. Implement the fix at the identified locations in pdd/agentic_change_orchestrator.py
3. Verify both unit and E2E tests pass with your fix
4. Run full test suite to check for regressions
5. Mark the PR as ready for review

PDD Fix Command

To auto-fix this bug using PDD:

cd /Users/gregtanaka/Documents/pdd_cloud/pdd/.pdd/worktrees/fix-issue-448
pdd --force fix --loop --max-attempts 5 --verification-program context/agentic_change_orchestrator_example.py prompts/agentic_change_orchestrator_python.prompt pdd/agentic_change_orchestrator.py tests/test_e2e_issue_448_step5_keyerror.py

Fix Location Summary

Add preprocess(..., double_curly_brackets=True) before calling .format() at these 4 locations in pdd/agentic_change_orchestrator.py:

• Line 598 (main step loop)
• Line 713 (step 11)
• Line 727 (step 12)
• Line 790 (step 13)

Reference implementation: pdd/agentic_architecture_orchestrator.py:299-302

---

Investigation complete. A draft PR with failing tests has been created and linked to this issue.

[gltanaka]

Fix Complete

The fix for this issue (and all related issues #319, #373, #376, #392) has been implemented.

The solution adds preprocess(..., double_curly_brackets=True) before .format() calls in the orchestrators, following the pattern already working in agentic_architecture_orchestrator.py.

Tests passing:

• 32 cherry-picked E2E tests for issues pdd change: failure at Step 5 #319, pdd change: KeyError '\n "type"' in Step 5 prompt template due to unescaped JSON in included prompting_guide.md #373, Message: Context missing key for step 5: '"url"' when doing pdd change #448
• 76 orchestrator unit tests
• 7 regression tests for issue Race Condition in LLM Cost Tracking Causes Data Corruption #375 (PDD tag handling)

Branch: promo-code-bugs