Skip to content

Fix #1198: warn on unknown keys in .pddrc with full schema validation - #1217

Merged
gltanaka merged 18 commits into
promptdriven:mainfrom
sohni-tagirisa:fix/1198-pddrc-unknown-keys
May 28, 2026
Merged

Fix #1198: warn on unknown keys in .pddrc with full schema validation#1217
gltanaka merged 18 commits into
promptdriven:mainfrom
sohni-tagirisa:fix/1198-pddrc-unknown-keys

Conversation

@sohni-tagirisa

@sohni-tagirisa sohni-tagirisa commented May 26, 2026

Copy link
Copy Markdown
Collaborator

Closes #1198.

Problem

.pddrc accepts unknown keys silently. Typos, stale documentation, and leftover keys from older schemas are dropped without warning, and PDD falls back to built-in defaults. Per the issue, this costs users ~1 hour of debugging per affected case.

Fix

Add schema-based validation in _load_pddrc_config that emits a UserWarning for any unknown key at the root, context, or defaults levels. Warning format matches the issue's specification: WARNING: .pddrc contains unknown key 'X' at path 'Y', ignored. Run 'pdd setup' to regenerate.

Unknown keys are reported but not rejected, so stale configs continue to load — avoids breaking users on upgrade. Could be promoted to errors in a future release.

Schema

Known keys by nesting level:

  • Root: version, contexts
  • Context: paths, defaults
  • Defaults: generate_output_path, test_output_path, example_output_path, prompts_dir, default_language, target_coverage, strength, temperature, budget, max_attempts, outputs

Schema verified by tracing pdd/construct_paths.py's _resolve_config_hierarchy to identify keys that are actually consumed.

Tests

7 new tests in tests/test_construct_paths.py:

  • Unknown root key emits warning
  • Unknown context key emits warning
  • Unknown defaults key emits warning
  • Clean config emits no warnings
  • Warning message format matches issue specification
  • Validator warns on auto_deps_csv_path (regression test)
  • Validator warns on prompt_path (regression test)

Findings surfaced by the validator

Running the validator against the codebase caught four real issues:

1. Test fixtures with incorrectly-nested keys

Two tests in TestConstructPathsResolutionModeParameter wrote .pddrc files with path keys directly under the context block (e.g., contexts.default.generate_output_path) rather than inside defaults. Dead config — the resolver ignores those keys at the context level. Fixed in this PR.

2. auto_deps_csv_path prescribed by templates but not consumed

PDD's template at pdd/templates/generic/generate_pddrc_YAML.prompt prescribed auto_deps_csv_path in every context across 9 occurrences. The repo's own production .pddrc and examples/prompts_linter/.pddrc (5 contexts each) followed this convention, and the example README instructed users to add the key. But _resolve_config_hierarchy never reads it — auto_deps_main falls back to the default "project_dependencies.csv".

Per PR review by @gltanaka: removed from the validator's allow-list, from the template (9 occurrences), from the prompt's schema doc, and from examples/prompts_linter/.pddrc and README.md (10 occurrences total). Following up with a separate issue for the actual wiring work (~6 touchpoints in path-resolution machinery).

3. match fixture in test_find_prompt_file.py:424

Validator surfaced contexts.backend-utils.match as an unknown key. Investigation showed match is not a real PDD schema key (the test's assertion only verifies prompts_dir). Removed from the fixture.

4. prompt_path documented but not consumed

Per PR second review by @gltanaka: prompt_path was in _PDDRC_DEFAULTS_KEYS because construct_paths_python.prompt documented it as an alias for prompts_dir, but _resolve_config_hierarchy never implemented the documented resolution chain. Same class of silent-ignore as auto_deps_csv_path.

Took the 'remove' option: stripped prompt_path from _PDDRC_DEFAULTS_KEYS, removed the alias claim from construct_paths_python.prompt (the prompt was the source-of-truth bug, not the code), and added a regression test. PDD_PROMPT_PATH env var is unaffected — only the .pddrc key alias was removed.

CI

Local filtered suite passes with the same failure count as main (verified with git stash + branch checkout). No new unknown key warnings emitted by the validator anywhere in the suite. Loading the repo's checked-in .pddrc and examples/prompts_linter/.pddrc both produce zero warnings.

Files changed

  • pdd/construct_paths.py — validator function, schema constants, call site
  • pdd/prompts/construct_paths_python.prompt — validation requirement documented, removed prompt_path alias claim
  • pdd/templates/generic/generate_pddrc_YAML.prompt — removed auto_deps_csv_path prescription
  • examples/prompts_linter/.pddrc — removed auto_deps_csv_path
  • examples/prompts_linter/README.md — removed auto_deps_csv_path instructions
  • tests/test_construct_paths.py — 7 new tests + 2 fixture corrections
  • tests/test_find_prompt_file.py — 1 fixture correction (match key)

Tests document the desired behavior: _load_pddrc_config should emit
UserWarning for unknown keys at the root, context, or defaults levels,
with a message format pointing users to 'pdd setup' for regeneration.

Per CONTRIBUTING.md red/green workflow: these tests fail without the
fix and will pass with it.
Adds schema-based validation in _load_pddrc_config. Emits a UserWarning
for any unknown key at the root, context, or defaults level, with the
format:

    WARNING: .pddrc contains unknown key 'X' at path 'Y', ignored.
    Run 'pdd setup' to regenerate.

Previously, unknown keys (typos, stale documentation, leftover keys
from older schemas) were silently dropped and PDD fell back to built-in
defaults with no signal to the user. Per issue promptdriven#1198, users were losing
roughly an hour of debugging per affected case.

Chose UserWarning over raising to avoid breaking users with stale
configs on upgrade. Could be promoted to errors in a future release.
)

The two cwd_mode fixtures in TestConstructPathsResolutionModeParameter
wrote .pddrc files with path keys directly under the context block
(contexts.default.generate_output_path) rather than inside defaults
(contexts.default.defaults.generate_output_path).

PDD's actual config resolution path (_get_context_config returns
defaults dict, _resolve_config_hierarchy reads from there) ignores
those keys when placed at the context level. The tests still passed
because their assertions only checked that output paths landed
somewhere under CWD, not whether the .pddrc values were actually
honored.

The validation added in the previous commit surfaced this. Fixed
both fixtures to use the documented nested form.
PDD's own template (templates/generic/generate_pddrc_YAML.prompt)
prescribes this key for every context, so it's part of PDD's actual
config shape and must be accepted by the validator.

Note: a grep of the codebase shows this key is referenced in code only
as a Python function parameter (e.g., auto_deps_main.py:27, call sites
in sync_orchestration.py, pin_example_hack.py, commands/maintenance.py),
never read from context_config. The unknown-key validator surfaced
this discrepancy: the template tells users to put auto_deps_csv_path
in every context, but no .pddrc resolver code consumes it. Worth a
separate issue to determine whether the template is wrong or the
resolver is missing functionality. Adding to schema unblocks the
warning meanwhile.
…mptdriven#1198)

Adds the unknown-key validation requirement to construct_paths_python.prompt
so the prompt (PDD doctrine: source of truth) reflects the new behavior
implemented in pdd/construct_paths.py. Lists the full known-key schema at
each nesting level (root, context, defaults) including the auto_deps_csv_path
key surfaced by running the validator against PDD's own .pddrc.

Per CONTRIBUTING.md: 'If editing code directly, run pdd update to sync
changes back into prompts.' Note: pdd update was attempted first but
crashed with [Errno 63] File name too long when constructing the
output path — a separate bug worth filing. Falling back to hand-edit
per CONTRIBUTING's 'maintainers may help sync' guidance.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@gltanaka gltanaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes rather than merging as-is.

The validator itself is needed for #1198; unknown .pddrc keys currently fail silently. The blocker is that this implementation whitelists and documents auto_deps_csv_path as a valid defaults key (pdd/construct_paths.py:132-145, pdd/prompts/construct_paths_python.prompt:51-54) while the runtime still does not consume that key from .pddrc.

Current flow:

  • _resolve_config_hierarchy only resolves the existing path/model keys and does not include auto_deps_csv_path (pdd/construct_paths.py:530-542).
  • generate_output_paths.COMMAND_OUTPUT_KEYS['auto-deps'] only includes output, so no csv output path is produced (pdd/generate_output_paths.py:18-33).
  • auto_deps_main then reads output_file_paths.get("csv", "project_dependencies.csv"), so the .pddrc key remains silently ignored (pdd/auto_deps_main.py:56-73).

Required changes before merge:

  1. Either wire auto_deps_csv_path through config resolution/output path generation and add regression coverage proving .pddrc affects the auto-deps CSV path, or remove it from the allow-list/template so users get the unknown-key warning instead of another silent ignore.
  2. Clean the new trailing whitespace reported by git diff --check at pdd/construct_paths.py:206.

Tests I ran: python -m pytest tests/test_construct_paths.py -q passed; git diff --check origin/main...HEAD failed on the trailing whitespace above.

@gltanaka

gltanaka commented May 26, 2026

Copy link
Copy Markdown
Contributor

Additional required cleanup from the broader non-integration run: the new validator emits a warning for contexts.backend-utils.match from tests/test_find_prompt_file.py:424. If match is an unsupported/stale .pddrc shape, please fix that fixture; if it is intended to be supported, add it to the schema and cover the resolver behavior. Either way, the validator change should not leave the suite with a new unknown-key warning.

…promptdriven#1198)

Per maintainer review on promptdriven#1217, removing the option to whitelist
auto_deps_csv_path. The validator's purpose is to surface silent
ignores, and this key is exactly such a case: prescribed by
pdd/templates/generic/generate_pddrc_YAML.prompt but never read by
_resolve_config_hierarchy or _get_context_config.

Changes:
- Remove auto_deps_csv_path from _PDDRC_DEFAULTS_KEYS in construct_paths.py
- Remove all 9 references to auto_deps_csv_path from generate_pddrc_YAML.prompt
  (6 YAML example lines, 3 documentation/instruction lines)
- Update construct_paths_python.prompt's schema documentation to match
- Strip trailing whitespace at construct_paths.py:206

Wiring auto_deps_csv_path through config resolution properly is a
separate concern that touches ~6 touchpoints in path-resolution
machinery (_resolve_config_hierarchy, COMMAND_OUTPUT_KEYS,
DEFAULT_FILENAMES, ENV_VAR_MAP, and auto_deps_main precedence logic).
Will file a follow-up issue.
….py (promptdriven#1198)

Per PR promptdriven#1217 review: validator surfaced an unknown-key warning for
contexts.backend-utils.match at tests/test_find_prompt_file.py:424.

Investigation: the 'match' key is not a real PDD schema key, and the
test's assertion only verifies that the prompts_dir context prefix is
honored — the 'match' entry was dead config. Removed it from the
fixture; the test still passes because the assertion never depended
on 'match'.

Same pattern as the cwd_mode fixture corrections in e8398bf.
…romptdriven#1198)

Lock in PR review feedback: if anyone re-adds auto_deps_csv_path to
_PDDRC_DEFAULTS_KEYS without wiring it through the resolver, or if
the template re-introduces the prescription, this test fails.

@gltanaka gltanaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the earlier review. The original blockers are fixed: auto_deps_csv_path was removed from _PDDRC_DEFAULTS_KEYS, the generator template no longer emits it, the match fixture was cleaned up, and git diff --check is now clean.

I still would not merge this current head because the validator now exposes checked-in stale config/docs and another allow-listed silent ignore:

  1. examples/prompts_linter/.pddrc still contains auto_deps_csv_path in five contexts, and examples/prompts_linter/README.md still instructs users to add the same key. Loading the repo's checked-in .pddrc files on this branch emits five new warnings from that example config. Please remove/update those entries too, otherwise a public example in the repo immediately warns under the new validator.

  2. prompt_path is still included in _PDDRC_DEFAULTS_KEYS, but _resolve_config_hierarchy does not resolve it; it only resolves prompts_dir and the other existing keys. I verified with a .pddrc containing defaults.prompt_path: configured_prompts: _load_pddrc_config emits no warning, but resolve_effective_config() returns only default_language and _matched_context. This is the same class of silent-ignore hole that was fixed for auto_deps_csv_path. Please either wire prompt_path through the resolver as the documented alias for prompts_dir with regression coverage, or remove it from the allow-list/schema docs until it is actually consumed.

Focused verification I ran:

  • git diff --check origin/main...HEAD: passed
  • python -m pytest tests/test_construct_paths.py -q: 111 passed
  • python -m pytest tests/test_find_prompt_file.py::TestPddrcPrefixForNewModules::test_new_module_respects_pddrc_prompts_dir_prefix -q: 1 passed
  • loading all checked-in .pddrc files still emits auto_deps_csv_path warnings from examples/prompts_linter/.pddrc

…1198)

Per PR promptdriven#1217 second review: loading the repo's checked-in .pddrc files
on this branch emitted 5 unknown-key warnings from
examples/prompts_linter/.pddrc, and the README still instructed users
to add the same key.

Since the parent commit removed auto_deps_csv_path from the schema
(no longer in _PDDRC_DEFAULTS_KEYS) and from the generator template,
the public example must also be cleaned up — otherwise the example
immediately warns under the new validator the moment someone tries it.

Removed all 5 occurrences from examples/prompts_linter/.pddrc and all
5 corresponding lines from examples/prompts_linter/README.md.
)

Per PR promptdriven#1217 second review: prompt_path was included in
_PDDRC_DEFAULTS_KEYS but _resolve_config_hierarchy never resolved it.
Same class of silent-ignore hole that was fixed for auto_deps_csv_path.

Greg's verification showed a .pddrc containing defaults.prompt_path:
configured_prompts emits no warning, but resolve_effective_config()
returns only default_language and _matched_context — the key was
documented as an alias but never actually wired through.

Took the 'remove' option:

- Removed 'prompt_path' from _PDDRC_DEFAULTS_KEYS in construct_paths.py
- Removed the alias claim from construct_paths_python.prompt:
  - Line 37: '.pddrc supports prompt_path as an alias for the prompt
    directory' — claim was false, prompt was the source-of-truth bug
  - Line 38: CLI override line that referenced prompt_path
  - Line 51: Removed prompt_path from the schema doc list
  - Line 91: Cleaned up the PDD_PROMPT_PATH / prompt_path discovery
    rules sentence — kept PDD_PROMPT_PATH (env var still respected),
    dropped the prompt_path reference

The validator now correctly surfaces prompt_path as an unknown key.
PDD_PROMPT_PATH env var is unaffected — only the .pddrc key alias
was removed.
@sohni-tagirisa

Copy link
Copy Markdown
Collaborator Author

Pushed updates addressing both points from your second review.

1. examples/prompts_linter cleanup (a9429e12) — removed all 5 occurrences of auto_deps_csv_path from examples/prompts_linter/.pddrc and all 5 corresponding lines from the README.

2. prompt_path (197ac26f, ec741626) — took the 'remove' option. Stripped prompt_path from _PDDRC_DEFAULTS_KEYS and from construct_paths_python.prompt's alias documentation (the prompt was the source-of-truth bug — it claimed an aliasing system that was never implemented). Added a regression test asserting the validator warns on the key. PDD_PROMPT_PATH env var is unaffected.

Verified locally with your exact checks:

  • git diff --check origin/main...HEAD → clean
  • python -m pytest tests/test_construct_paths.py -q → 114 passed
  • python -m pytest tests/test_find_prompt_file.py::TestPddrcPrefixForNewModules::test_new_module_respects_pddrc_prompts_dir_prefix -q → 1 passed
  • Loading .pddrc and examples/prompts_linter/.pddrc — both produce zero warnings

PR description updated. Ready for re-review.

@sohni-tagirisa sohni-tagirisa self-assigned this May 27, 2026
@sohni-tagirisa sohni-tagirisa added the pdd-connect PDD: start a connect session label May 27, 2026
@prompt-driven-github

Copy link
Copy Markdown
Contributor

🔗 Connect Session Starting!

Session ID: 052250ce-2889-4731-a769-8e112ecfb053
Triggered by: @sohni-tagirisa
You can interact with this session from the PDD Cloud web UI.
The session will auto-commit and create a PR on shutdown.

…riven#1198)

Lock in PR review feedback: if anyone re-adds prompt_path to
_PDDRC_DEFAULTS_KEYS without wiring it through _resolve_config_hierarchy
as a prompts_dir alias, this test fails.
@sohni-tagirisa
sohni-tagirisa force-pushed the fix/1198-pddrc-unknown-keys branch from 6673a70 to 4f427fb Compare May 27, 2026 21:25

@gltanaka gltanaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update. The .pddrc validator issues from the previous review look addressed now: examples/prompts_linter no longer carries auto_deps_csv_path, prompt_path is no longer documented as supported without wiring, and the focused tests pass locally.

I still would not merge this head because unrelated generated/local artifacts are now included in the PR:

  • pdd-env/ commits a local virtualenv, including activation scripts, console entrypoints, Python symlinks, and pdd-env/pyvenv.cfg with contributor-local paths such as /Users/sohnitagirisa/pdd/pdd-2/pdd-env.
  • git diff --check origin/main...HEAD fails on the committed virtualenv files, starting with trailing whitespace throughout pdd-env/bin/Activate.ps1.
  • error_log.txt is added and appears to be a local generated test/debug artifact.
  • .gitignore removes the committed .pdd/meta/... exceptions; that looks unrelated to issue #1198 and should not be included unless there is a deliberate repo-level reason.

Required changes before merge:

  1. Remove pdd-env/ from the PR entirely and keep local virtualenvs ignored.
  2. Remove error_log.txt unless this PR intentionally needs it and the purpose is documented.
  3. Revert the unrelated .gitignore deletion, or split/explain it separately if it is intentional.
  4. Keep the PR scoped to the .pddrc validation, template/example cleanup, and regression tests.

Local checks run on head 34034fb4:

  • python -m pytest tests/test_construct_paths.py -q -> 114 passed
  • python -m pytest tests/test_find_prompt_file.py::TestPddrcPrefixForNewModules::test_new_module_respects_pddrc_prompts_dir_prefix -q -> 1 passed
  • git diff --check origin/main...HEAD -> fails because of the committed pdd-env/ files

@sohni-tagirisa sohni-tagirisa added pdd-bug PDD: fix a bug pdd-opus and removed pdd-connect PDD: start a connect session labels May 27, 2026
@prompt-driven-github

Copy link
Copy Markdown
Contributor

🚀 Job Queued!

Job ID: B2wflTGaXVpDGXr4GhzM
Triggered by: @sohni-tagirisa
Label: bug

View Live Progress

@sohni-tagirisa sohni-tagirisa removed pdd-opus pdd-bug PDD: fix a bug labels May 27, 2026
@prompt-driven-github

prompt-driven-github Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 1: Duplicate Check

Status: No duplicates found

Search Performed

  • Searched for: pddrc unknown keys, schema validation pddrc, warn unknown key config
  • Issues reviewed: 6 candidates across open and closed issues

Findings

No duplicate issues were found. Issue #1217 is a pull request (branch sohni-tagirisa:fix/1198-pddrc-unknown-keys) that explicitly fixes the original bug report #1198 (.pddrc silently ignores unknown keys; no schema validation).

Since the original issue #1198 remains open and unresolved, this PR should proceed through the investigation workflow normally.


Proceeding to Step 2: Documentation Check

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 2: Documentation Check

Status: Confirmed Bug

Documentation Reviewed

  • README.md — §"Project Configuration File (.pddrc)" (lines ~3123–3205): documents the valid set of keys (version, contexts, paths, defaults, and the defaults sub-keys) but says nothing about what happens when unknown keys are present.
  • pdd/construct_paths.py_load_pddrc_config (lines 128–145): current implementation loads YAML, asserts the root is a dict and that contexts is present, then returns the config as-is. No validation of unknown keys; they are silently dropped.
  • CONTRIBUTING.md — no mention of the silent-drop behavior.
  • docs/path_resolution_resolver.md — no mention of unknown-key handling.

Findings

The README documents a precise schema for .pddrc (root keys version/contexts, context keys paths/defaults, and a fixed set of defaults sub-keys). There is no documentation stating that unknown keys are intentionally accepted and silently ignored. The absence of any warning when an unknown key is present contradicts the documented schema and causes a real usability problem: typos and stale config keys are silently dropped, causing PDD to fall back to built-in defaults with no diagnostic output.

The behavior is a confirmed bug — the code does not enforce the documented schema, and there is no documented intent for keys outside the schema to be accepted silently.

Issue #1217 is the corresponding fix PR (branch sohni-tagirisa:fix/1198-pddrc-unknown-keys), targeting the original report in #1198.


Proceeding to Step 3: Triage

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 3: Triage

Status: Fast-Track

Pre-Diagnosed Issue Detected

The issue author has provided a detailed root cause analysis:

Identified Root Cause:
The bug is in _load_pddrc_config in pdd/construct_paths.py (lines 128–145). The function only validates that the root is a dict and that contexts exists, then returns the config as-is. Unknown keys at any nesting level (root, context, or defaults) are silently dropped, causing users to lose configuration without any diagnostic output. The fix adds schema-based validation using three constant sets (_PDDRC_ROOT_KEYS, _PDDRC_CONTEXT_KEYS, _PDDRC_DEFAULTS_KEYS) and a validator function that emits UserWarning for any key not in the schema.

Evidence Quality:

  • File paths with line numbers: Yes
    • pdd/construct_paths.py_load_pddrc_config (lines 128–145), new validator function and schema constants
    • pdd/prompts/construct_paths_python.prompt — removed prompt_path alias claim
    • pdd/templates/generic/generate_pddrc_YAML.prompt — removed 9 occurrences of auto_deps_csv_path
    • examples/prompts_linter/.pddrc and examples/prompts_linter/README.md — removed auto_deps_csv_path
    • tests/test_construct_paths.py — 7 new tests + 2 fixture corrections
    • tests/test_find_prompt_file.py:424 — fixture correction (removed stale match key)
  • Causal explanation: Yes — _load_pddrc_config performs no schema enforcement; any key outside the documented schema passes silently and falls back to built-in defaults
  • Reproduction evidence: Yes — 7 new unit tests covering unknown root/context/defaults keys, clean-config no-warning case, warning message format, and two regression tests (auto_deps_csv_path, prompt_path)

Summary

This is a PR (#1217) implementing a fix for confirmed bug #1198. The root cause is fully diagnosed: _load_pddrc_config in pdd/construct_paths.py lacks any schema enforcement, silently dropping unknown keys. The PR introduces schema constants and a validator, has 7 new tests passing (114 total in the construct_paths suite), and verifies that zero unknown-key warnings are emitted by the validator against the repo's own .pddrc and examples/prompts_linter/.pddrc. All reviewer-requested cleanup (the match fixture at line 424, auto_deps_csv_path, and prompt_path) has been addressed. Diagnosis steps can be skipped.


Fast-tracking to Step 6: Root Cause Analysis (skipping Steps 4-5)

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 6: Root Cause Analysis

Root Cause Identified: Yes

Summary

The bug is in _load_pddrc_config in pdd/construct_paths.py (lines 128–145), which only validates that the parsed YAML is a dict containing a contexts key, then returns the entire config as-is with no schema enforcement. Unknown keys at the root level, context level, or defaults level are silently discarded — the resolver never reads them — so a typo like promts_dir or a stale key like auto_deps_csv_path causes PDD to silently fall back to built-in defaults with zero diagnostic output. All five callers of _load_pddrc_config (list_available_contexts, detect_context_for_file, get_tests_dir_from_config, resolve_effective_config, construct_paths) inherit this silent-drop behavior. The fix is localized to one place: add schema constants and a validator call inside _load_pddrc_config so the warning is emitted once on load, regardless of which call site triggered the load.

Technical Details

  • Location: pdd/construct_paths.py:128–145
  • Problematic Code:
def _load_pddrc_config(pddrc_path: Path) -> Dict[str, Any]:
    """Load and parse .pddrc configuration file."""
    try:
        with open(pddrc_path, 'r', encoding='utf-8') as f:
            config = yaml.safe_load(f)
        
        if not isinstance(config, dict):
            raise ValueError(f"Invalid .pddrc format: expected dictionary at root level")
        
        # Validate basic structure
        if 'contexts' not in config:
            raise ValueError(f"Invalid .pddrc format: missing 'contexts' section")
        
        return config   # ← returned without any unknown-key checks
    except yaml.YAMLError as e:
        raise ValueError(f"YAML syntax error in .pddrc: {e}")
    except Exception as e:
        raise ValueError(f"Error loading .pddrc: {e}")
  • Why it fails: After the two structural guards (is-dict, has-contexts), the config is returned verbatim. _resolve_config_hierarchy only iterates over a fixed allow-list of keys (generate_output_path, test_output_path, example_output_path, prompts_dir, default_language, target_coverage, strength, temperature, budget, max_attempts, outputs), so any key outside that list is implicitly dropped with no warning emitted.

External Research

  • Search terms: pyyaml unknown keys warning, python UserWarning unknown config key, YAML schema validation unknown keys
  • Sources checked: PyYAML docs, Python warnings module docs, GitHub issues on YAML config validation patterns
  • Relevant findings: PyYAML safe_load intentionally accepts any key — schema enforcement is the application's responsibility. The standard Python pattern for soft validation (warn but don't reject) is warnings.warn(..., UserWarning). No external dependency is needed.

Repository History

Experiments Performed

  1. Read _load_pddrc_config in full — confirmed no unknown-key checks exist anywhere in the function body or its callers.
  2. Read _resolve_config_hierarchy (lines 456–495) — confirmed the allow-list of consumed keys: generate_output_path, test_output_path, example_output_path, prompts_dir, default_language, target_coverage, strength, temperature, budget, max_attempts, outputs. Anything else is dropped silently.
  3. Grep for auto_deps_csv_path — confirmed 9 occurrences in pdd/templates/generic/generate_pddrc_YAML.prompt and 5 in examples/prompts_linter/.pddrc still prescribing the key; _resolve_config_hierarchy never reads it.
  4. Read tests/test_find_prompt_file.py:419–428 — confirmed match is nested as contexts.backend-utils.match (a context-level key), not a real schema key; paths is nested under match (contexts.backend-utils.match.paths) and thus never consumed by _match_path_to_contexts, which reads context_config.get('paths', []) directly.
  5. Read tests/test_construct_paths.py:2526–2531 and 2586–2591 — confirmed two TestConstructPathsResolutionModeParameter fixtures write generate_output_path, test_output_path, example_output_path directly under the context block rather than under defaults; _get_context_config only reads context_config.get('defaults', {}), so those keys are dead config.
  6. Grep for prompt_path in pdd/prompts/construct_paths_python.prompt — confirmed lines 37–38 and 86 claim prompt_path is an alias for prompts_dir resolved by the config hierarchy, but _resolve_config_hierarchy has no such alias logic.

Fix Scope

  • Type: Localized — the unknown-key silencing happens at one load point (_load_pddrc_config), called by all 5 downstream callers. A single validator call inside that function covers all call sites with no per-caller changes.
  • Additional cleanup files (stale keys and fixture corrections): These are side-effects discovered by running the validator, not separate architectural concerns. They are mechanical removals, not structural changes.

Sibling Bugs

  • Variable reference audit:
    • _load_pddrc_config is written at construct_paths.py:128 (function definition).
    • Read (called) at: list_available_contexts (line 157), detect_context_for_file (line 417), get_tests_dir_from_config (line 598), resolve_effective_config (line 540), construct_paths (line 1028). All 5 call sites pass through the missing validation consistently — no inconsistency between call sites.
  • State symmetry check: No save/restore pattern. Config is loaded from disk and processed in memory; no checkpointing or serialization.
  • Independent bugs found:
    1. tests/test_find_prompt_file.py:424match key at contexts.backend-utils.match is not a valid context-level key; its child paths is never read by _match_path_to_contexts, making the fixture's path pattern dead config.
    2. tests/test_construct_paths.py:2526–2531generate_output_path, test_output_path, example_output_path placed directly under contexts.default instead of contexts.default.defaults; _get_context_config never reads them.
    3. tests/test_construct_paths.py:2586–2591 — Same incorrect nesting as (2), second fixture in the same test class.
    4. pdd/prompts/construct_paths_python.prompt:37–38, 86 — Documents prompt_path as a .pddrc alias for prompts_dir, but no alias resolution exists in _resolve_config_hierarchy.
    5. pdd/templates/generic/generate_pddrc_YAML.prompt (9 occurrences) — Prescribes auto_deps_csv_path in every context; never consumed by the resolver.
    6. examples/prompts_linter/.pddrc (5 occurrences) and examples/prompts_linter/README.md (5 occurrences) — Follow the template's stale prescription.

Proposed Fix Validation

  • Issue proposes a fix: Yes
  • Scope classification: SCOPE_MATCH
  • Issue-proposed scope:
    1. pdd/construct_paths.py — validator function + schema constants + call site in _load_pddrc_config
    2. pdd/prompts/construct_paths_python.prompt — remove prompt_path alias claim
    3. pdd/templates/generic/generate_pddrc_YAML.prompt — remove auto_deps_csv_path (9 occurrences)
    4. examples/prompts_linter/.pddrc — remove auto_deps_csv_path (5 occurrences)
    5. examples/prompts_linter/README.md — remove auto_deps_csv_path instructions (5 occurrences)
    6. tests/test_construct_paths.py — 7 new tests + 2 fixture corrections (lines 2526–2531 and 2586–2591)
    7. tests/test_find_prompt_file.py — remove match key from fixture (line 424)
  • Step 6 full scope: Matches the 7 items above — independent analysis found no additional affected locations.
  • Expansion items: None.

Fix Location

  • File(s) to modify: pdd/construct_paths.py, pdd/prompts/construct_paths_python.prompt, pdd/templates/generic/generate_pddrc_YAML.prompt, examples/prompts_linter/.pddrc, examples/prompts_linter/README.md, tests/test_construct_paths.py, tests/test_find_prompt_file.py
  • Why this location: _load_pddrc_config is the single parse-and-validate entry point for all .pddrc consumers. All 5 callers delegate to it, so adding the validator there covers 100% of load paths with one change.
  • Existing pattern: Python's warnings.warn(msg, UserWarning) is the standard soft-validation idiom used elsewhere in the codebase for non-fatal config issues. The existing _resolve_config_hierarchy allow-list already defines exactly which keys are consumed — the schema constants (_PDDRC_ROOT_KEYS, _PDDRC_CONTEXT_KEYS, _PDDRC_DEFAULTS_KEYS) are a natural complement.
  • Fix approach: Add three frozenset constants defining the allowed keys at each nesting level, then add a _validate_pddrc_keys(config, pddrc_path) helper that iterates root keys, context keys, and defaults keys, calling warnings.warn for each unknown key. Call _validate_pddrc_keys from _load_pddrc_config after the structural checks pass, before returning.

Proceeding to Step 7: Prompt Classification

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 7: Prompt Classification

Classification: Prompt Defect

DEFECT_TYPE: prompt
PROMPT_FIXED: pdd/prompts/construct_paths_python.prompt

Analysis

Two prompt defects were identified in pdd/prompts/construct_paths_python.prompt:

  1. False prompt_path alias claim (primary prompt defect): The prompt explicitly stated .pddrc supports 'prompt_path' as an alias for the prompt directory and CLI 'prompts_dir' overrides CLI 'prompt_path' if both are provided. However, _resolve_config_hierarchy iterates over a fixed config_keys dict that never included prompt_path—the code correctly did not implement this alias. The prompt was wrong, not the code. Per the PR analysis: "the prompt was the source-of-truth bug — it claimed an aliasing system that was never implemented."

  2. Missing schema validation requirement (secondary prompt defect): The prompt never specified that _load_pddrc_config should emit UserWarning for unknown keys. Without this requirement in the prompt, the generated code had no contract to enforce, causing unknown keys to be silently dropped.

Evidence

  • Prompt specified: prompt_path as a supported .pddrc alias + no schema validation requirement
  • Code implements: No prompt_path alias in _resolve_config_hierarchy (correct behavior); no unknown-key warnings in _load_pddrc_config
  • User expects: prompt_path NOT a valid .pddrc key (should warn); all unknown keys at root/context/defaults levels emit UserWarning

Prompt Change Made

File: pdd/prompts/construct_paths_python.prompt

Before (Requirement 1):

1. Configuration Hierarchy: CLI options > .pddrc context > environment variables > defaults.
   - Environment variables: PDD_PROMPT_PATH takes precedence over PDD_PROMPTS_DIR.
   - .pddrc supports 'prompt_path' as an alias for the prompt directory.
   - CLI 'prompts_dir' overrides CLI 'prompt_path' if both are provided.
   - Implement helpers: _find_pddrc_file, _load_pddrc_config, ...
   ...

After (Requirement 1):

1. Configuration Hierarchy: CLI options > .pddrc context > environment variables > defaults.
   - Environment variables: PDD_PROMPT_PATH takes precedence over PDD_PROMPTS_DIR.
   - Implement helpers: _find_pddrc_file, _load_pddrc_config, ...
   ...
   - `_load_pddrc_config` MUST emit a `UserWarning` for every unknown key at the root,
     context, or defaults nesting levels using the format:
     `WARNING: .pddrc contains unknown key 'X' at path 'Y', ignored. Run 'pdd setup' to regenerate.`
     Unknown keys are warned but not rejected; the config loads normally.
     Known root keys: `version`, `contexts`.
     Known context keys: `paths`, `defaults`.
     Known defaults keys: `generate_output_path`, `test_output_path`, `example_output_path`,
     `prompts_dir`, `default_language`, `target_coverage`, `strength`, `temperature`,
     `budget`, `max_attempts`, `outputs`.

Conclusion

The prompt specification was incorrect on two counts: it falsely documented a prompt_path alias that was never implemented, and it omitted the schema validation contract entirely. Both have been corrected. Proceeding with test generation based on the corrected specification.


Proceeding to Step 8: Test Plan

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 8: Test Plan

Existing Test Coverage

  • Test file: tests/test_construct_paths.py (3807 lines, pytest)
  • Current coverage: Exercises _load_pddrc_config indirectly via mocked call sites (all callers patch _load_pddrc_config directly). Two integration tests call the real function against a live .pddrc (test_detect_context_real, test_get_context_config_real). Warnings channel is untested.
  • Gap: No tests for the schema validator — the UserWarning emission for unknown root, context, or defaults keys does not exist yet and is entirely uncovered.

Proposed Tests

Test 1: Unknown root key emits UserWarning

  • Input: _load_pddrc_config called with a temp .pddrc containing {"version": "1", "contexts": {"default": {}}, "typo_key": "value"}
  • Expected: pytest.warns(UserWarning) fires with message matching "unknown key 'typo_key'"
  • Actual (before fix): No warning emitted; typo_key silently dropped

Test 2: Unknown context key emits UserWarning

  • Input: .pddrc with contexts.default containing {"paths": [], "stale_key": "oops"}
  • Expected: pytest.warns(UserWarning) fires; message contains "unknown key 'stale_key'" and path "contexts.default"
  • Actual (before fix): Silent drop

Test 3: Unknown defaults key emits UserWarning

  • Input: .pddrc with contexts.default.defaults containing {"prompts_dir": "prompts", "unknown_option": "value"}
  • Expected: pytest.warns(UserWarning) fires; message contains "unknown key 'unknown_option'" and path "contexts.default.defaults"
  • Actual (before fix): Silent drop

Test 4: Clean config emits no warnings

  • Input: .pddrc using only known keys — root: version, contexts; context: paths, defaults; defaults: prompts_dir, generate_output_path
  • Expected: pytest.warns(UserWarning) block records zero warnings
  • Actual (before fix): Also no warnings — but this test guards against the fix over-warning

Test 5: Warning message matches issue specification exactly

  • Input: .pddrc with one unknown root key "bad_key"
  • Expected: Warning message string equals "WARNING: .pddrc contains unknown key 'bad_key' at path 'bad_key', ignored. Run 'pdd setup' to regenerate."
  • Actual (before fix): No warning issued; format not enforced

Test 6: Unknown key does not reject the config (load continues)

  • Input: .pddrc with unknown root key plus a valid contexts block
  • Expected: _load_pddrc_config still returns a dict containing "contexts" — config loads normally despite the warning
  • Actual (before fix): Config loads but no warning; behavior after fix must preserve backward-compatible loading

Test 7: Regression — auto_deps_csv_path warns

  • Input: .pddrc with contexts.default.defaults.auto_deps_csv_path set
  • Expected: pytest.warns(UserWarning) fires; message contains "unknown key 'auto_deps_csv_path'"
  • Rationale: auto_deps_csv_path was prescribed by 9 template occurrences but never consumed by _resolve_config_hierarchy. This regression test ensures the key stays out of the allow-list.

Test 8: Regression — prompt_path warns

  • Input: .pddrc with contexts.default.defaults.prompt_path set
  • Expected: pytest.warns(UserWarning) fires; message contains "unknown key 'prompt_path'"
  • Rationale: prompt_path was previously in _PDDRC_DEFAULTS_KEYS based on a false alias claim in the prompt; it was removed in the fix. PDD_PROMPT_PATH env var is unaffected.

Test 9: Multiple unknown keys produce separate warnings

  • Input: .pddrc with two unknown keys at different levels (e.g., root-level deprecated_flag and defaults-level old_output)
  • Expected: Two distinct UserWarning records, one per unknown key
  • Actual (before fix): Zero warnings; the fix must warn per-key, not collapse them

Test 10: match key at context level warns

  • Input: .pddrc with contexts.backend-utils containing {"defaults": {"prompts_dir": "prompts/backend/utils"}, "match": {"paths": ["backend/utils/"]}} — the exact shape from test_find_prompt_file.py:424
  • Expected: pytest.warns(UserWarning) fires; message contains "unknown key 'match'" at path "contexts.backend-utils"
  • Rationale: match is not in the context schema. The fixture correction in test_find_prompt_file.py removes the key; this test verifies the validator correctly flags it.

Test 11: Warning propagates through list_available_contexts

  • Input: Write a .pddrc with an unknown root key to tmpdir; call list_available_contexts(tmpdir)
  • Expected: pytest.warns(UserWarning) fires (validator runs through the real call stack, not a stub)
  • Rationale: _load_pddrc_config is called from multiple callers (construct_paths, list_available_contexts, detect_context_for_file, get_tests_dir_from_config, resolve_effective_config). This integration test verifies the warning is not filtered or suppressed anywhere in the real call chain.

Test Location

  • File: tests/test_construct_paths.py (append — preferred)
  • Framework: pytest with pytest.warns(UserWarning)

Notes

  • All 11 tests call the real _load_pddrc_config (or a real caller of it). No mocking of the function under test — the fix must produce the UserWarning for tests to pass.
  • Tests write temporary .pddrc files using tmpdir/tmp_path and import _load_pddrc_config directly alongside the existing imports.
  • Tests 1–10 are unit tests against _load_pddrc_config directly; Test 11 is a lightweight integration test via list_available_contexts to cover the call boundary.
  • Warning format string must be checked verbatim in Test 5 using str(w.message) or w.message.args[0] to catch format regressions.
  • The warnings.warn(..., UserWarning, stacklevel=2) stacklevel in the implementation should point to the call site; tests don't need to assert stacklevel, only the message content and category.

Proceeding to Step 9: Generate Test

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 9: Generated Test

Test File

tests/test_construct_paths.py (append)

Test Code

import warnings as _warnings_module
import yaml as _yaml_module


class TestPddrcSchemaValidation:
    """Tests for unknown-key validation in _load_pddrc_config (Issue #1198)."""

    def _write_pddrc(self, tmp_path, content):
        pddrc = tmp_path / ".pddrc"
        pddrc.write_text(_yaml_module.dump(content), encoding="utf-8")
        return pddrc

    def test_unknown_root_key_emits_warning(self, tmp_path): ...
    def test_unknown_context_key_emits_warning(self, tmp_path): ...
    def test_unknown_defaults_key_emits_warning(self, tmp_path): ...
    def test_clean_config_emits_no_warnings(self, tmp_path): ...
    def test_warning_message_format(self, tmp_path): ...
    def test_auto_deps_csv_path_emits_warning(self, tmp_path): ...
    def test_prompt_path_emits_warning(self, tmp_path): ...
    def test_multiple_unknown_keys_each_emit_warning(self, tmp_path): ...
    def test_unknown_key_does_not_reject_config(self, tmp_path): ...
    def test_construct_paths_triggers_validation_via_load(self, tmp_path, monkeypatch): ...
    def test_match_key_in_context_emits_warning(self, tmp_path): ...

What This Test Verifies

11 tests appended to tests/test_construct_paths.py as class TestPddrcSchemaValidation:

# Test Fails now?
1 test_unknown_root_key_emits_warning Yes — _load_pddrc_config emits 0 warnings
2 test_unknown_context_key_emits_warning Yes
3 test_unknown_defaults_key_emits_warning Yes
4 test_clean_config_emits_no_warnings No — guard against false positives after fix
5 test_warning_message_format Yes — no warning emitted to inspect
6 test_auto_deps_csv_path_emits_warning Yes — regression for stale key
7 test_prompt_path_emits_warning Yes — regression for removed alias
8 test_multiple_unknown_keys_each_emit_warning Yes
9 test_unknown_key_does_not_reject_config No — guard for non-rejecting behavior
10 test_construct_paths_triggers_validation_via_load Yes — caller chain not wired
11 test_match_key_in_context_emits_warning Yes — regression for fixture cleanup

Current result: 9 failed, 2 passed (1.09s)

The fix needs to add three schema-constant sets (_PDDRC_ROOT_KEYS, _PDDRC_CONTEXT_KEYS, _PDDRC_DEFAULTS_KEYS) and a validator called from _load_pddrc_config that emits UserWarning for any key not in those sets, with the message format: .pddrc contains unknown key 'X' at path 'Y', ignored. Run 'pdd setup' to regenerate.

Running the Test

python -m pytest tests/test_construct_paths.py::TestPddrcSchemaValidation -v

Proceeding to Step 10: Verification

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 9: Generated Test

Test File

tests/test_construct_paths.py (append)

Test Code

# ---------------------------------------------------------------------------
# Fix-location file cleanup tests (Issue #1198 / PR #1217)
# Tests for: examples/prompts_linter/.pddrc, pdd/templates/generic/generate_pddrc_YAML.prompt,
# and examples/prompts_linter/README.md
# ---------------------------------------------------------------------------

_REPO_ROOT = Path(__file__).parent.parent


def test_examples_prompts_linter_pddrc_no_auto_deps_csv_path():
    """examples/prompts_linter/.pddrc must not contain auto_deps_csv_path in any defaults block."""
    import yaml as _yaml
    pddrc_path = _REPO_ROOT / 'examples' / 'prompts_linter' / '.pddrc'
    config = _yaml.safe_load(pddrc_path.read_text(encoding='utf-8'))
    contexts_with_stale_key = [
        ctx_name for ctx_name, ctx_config in (config.get('contexts') or {}).items()
        if isinstance(ctx_config, dict)
        and 'auto_deps_csv_path' in (ctx_config.get('defaults') or {})
    ]
    assert not contexts_with_stale_key, (
        f'examples/prompts_linter/.pddrc still contains auto_deps_csv_path in contexts: {contexts_with_stale_key}'
    )


def test_generate_pddrc_yaml_template_no_auto_deps_csv_path_in_example_yaml():
    """pdd/templates/generic/generate_pddrc_YAML.prompt must not prescribe auto_deps_csv_path."""
    import yaml as _yaml
    template_path = _REPO_ROOT / 'pdd' / 'templates' / 'generic' / 'generate_pddrc_YAML.prompt'
    template_content = template_path.read_text(encoding='utf-8')
    marker_pos = template_content.find('EXAMPLE OUTPUT STRUCTURE')
    assert marker_pos != -1
    after_marker = template_content[marker_pos:]
    fence_start = after_marker.find('```yaml')
    yaml_body_start = fence_start + len('```yaml')
    fence_end = after_marker.find('```', yaml_body_start)
    example_config = _yaml.safe_load(after_marker[yaml_body_start:fence_end].strip())
    contexts_with_stale_key = [
        ctx_name for ctx_name, ctx_config in (example_config.get('contexts') or {}).items()
        if isinstance(ctx_config, dict)
        and 'auto_deps_csv_path' in (ctx_config.get('defaults') or {})
    ]
    assert not contexts_with_stale_key, (
        f'generate_pddrc_YAML.prompt example still prescribes auto_deps_csv_path in: {contexts_with_stale_key}'
    )


def test_examples_prompts_linter_readme_no_auto_deps_csv_path_in_yaml_snippets():
    """examples/prompts_linter/README.md must not show auto_deps_csv_path in YAML snippets."""
    import yaml as _yaml
    readme_path = _REPO_ROOT / 'examples' / 'prompts_linter' / 'README.md'
    readme_content = readme_path.read_text(encoding='utf-8')
    fence_start = readme_content.find('```yaml')
    yaml_body_start = fence_start + len('```yaml')
    fence_end = readme_content.find('```', yaml_body_start)
    snippet_config = _yaml.safe_load(readme_content[yaml_body_start:fence_end].strip())
    contexts_with_stale_key = [
        ctx_name for ctx_name, ctx_config in (snippet_config.get('contexts') or {}).items()
        if isinstance(ctx_config, dict)
        and 'auto_deps_csv_path' in (ctx_config.get('defaults') or {})
    ]
    assert not contexts_with_stale_key, (
        f'README.md YAML snippet still contains auto_deps_csv_path in: {contexts_with_stale_key}'
    )

What This Test Verifies

11 tests in TestPddrcSchemaValidation (previously added): Verify that _load_pddrc_config emits UserWarning for unknown keys at root, context, and defaults nesting levels. All fail because no validator exists yet in pdd/construct_paths.py.

3 new tests for missing fix-location files (added this step):

Test File Covered Failure Mechanism
test_examples_prompts_linter_pddrc_no_auto_deps_csv_path examples/prompts_linter/.pddrc File still has auto_deps_csv_path in all 5 contexts
test_generate_pddrc_yaml_template_no_auto_deps_csv_path_in_example_yaml pdd/templates/generic/generate_pddrc_YAML.prompt Template example YAML still has auto_deps_csv_path in all 5 contexts
test_examples_prompts_linter_readme_no_auto_deps_csv_path_in_yaml_snippets examples/prompts_linter/README.md README YAML snippet still has auto_deps_csv_path in all 5 contexts

Verification: 12/14 new tests fail on current code (2 are non-breaking guards that correctly pass both before and after the fix).

Running the Test

pytest tests/test_construct_paths.py::TestPddrcSchemaValidation -v
pytest tests/test_construct_paths.py::test_examples_prompts_linter_pddrc_no_auto_deps_csv_path -v
pytest tests/test_construct_paths.py::test_generate_pddrc_yaml_template_no_auto_deps_csv_path_in_example_yaml -v
pytest tests/test_construct_paths.py::test_examples_prompts_linter_readme_no_auto_deps_csv_path_in_yaml_snippets -v

Proceeding to Step 10: Verification

@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 10: Verification Complete

Test Execution

FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_unknown_root_key_emits_warning - AssertionError: Expected UserWarning mentioning 'typo_key' for unknown root key, got: []
FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_unknown_context_key_emits_warning - AssertionError: Expected UserWarning mentioning 'unknown_ctx_key', got: []
FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_unknown_defaults_key_emits_warning - AssertionError: Expected UserWarning mentioning 'stale_setting', got: []
FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_warning_message_format - AssertionError: Expected at least one UserWarning for unknown key 'bad_key' (0 >= 1)
FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_auto_deps_csv_path_emits_warning - AssertionError: Expected UserWarning for stale key 'auto_deps_csv_path', got: []
FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_prompt_path_emits_warning - AssertionError: Expected UserWarning for undocumented key 'prompt_path', got: []
FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_multiple_unknown_keys_each_emit_warning - AssertionError: Missing warning for context-level key 'unknown_ctx': []
FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_construct_paths_triggers_validation_via_load - AssertionError: construct_paths did not propagate the unknown-key UserWarning from _load_pddrc_config. Got: []
FAILED tests/test_construct_paths.py::TestPddrcSchemaValidation::test_match_key_in_context_emits_warning - AssertionError: Expected UserWarning for unknown context key 'match', got: []
FAILED tests/test_construct_paths.py::test_examples_prompts_linter_pddrc_no_auto_deps_csv_path - AssertionError: examples/prompts_linter/.pddrc still contains 'auto_deps_csv_path' in contexts: ['utils', 'cli', 'backend', 'frontend', 'default'].
FAILED tests/test_construct_paths.py::test_generate_pddrc_yaml_template_no_auto_deps_csv_path_in_example_yaml - AssertionError: generate_pddrc_YAML.prompt example YAML still prescribes 'auto_deps_csv_path' in contexts: ['utils', 'cli', 'backend', 'frontend', 'default'].
FAILED tests/test_construct_paths.py::test_examples_prompts_linter_readme_no_auto_deps_csv_path_in_yaml_snippets - AssertionError: examples/prompts_linter/README.md YAML snippet still contains 'auto_deps_csv_path' in contexts: ['utils', 'cli', 'backend', 'frontend', 'default'].
========================= 12 failed, 2 passed in 1.27s =========================

Verification Status

PASS: Test correctly detects the bug

All 12 failures are for the correct reason:

  • 9 schema-validation tests fail because _load_pddrc_config emits zero UserWarnings for any unknown key at root, context, or defaults nesting levels — the core bug.
  • 3 data-file tests fail because examples/prompts_linter/.pddrc, pdd/templates/generic/generate_pddrc_YAML.prompt, and examples/prompts_linter/README.md still contain the stale auto_deps_csv_path key.

Structural test check: PASS — No test uses inspect.getsource, hasattr, inspect.signature, or Python source-string scanning. All 9 warning tests call _load_pddrc_config() and assert on emitted UserWarnings (behavioral). The 3 data-file tests use yaml.safe_load on config/template/README files and check parsed key presence (data-file content check, not source scanning).

Mock cross-validation: PASS — The one mock in test_construct_paths_triggers_validation_via_load patches generate_output_paths to allow construct_paths to complete without writing files. This mock is orthogonal to config loading and warning emission — the validator fires before output path generation.

Summary

Step Result
Duplicate Check No duplicates — fixes open issue #1198
Documentation Confirmed bug — silent key drop undocumented
Triage Fast-tracked — full root cause in PR
API Research Skipped (fast-track)
Reproduction Skipped (fast-track)
Root Cause _load_pddrc_config lines 128–145, no schema enforcement
Prompt Classification Prompt defect — false prompt_path alias + missing validation contract
Test Plan 11 tests planned
Test Generation 12 tests generated (11 unit + 1 integration)
Verification PASS — 12 failures confirmed, all for correct reason

Bug Details

  • Location: pdd/construct_paths.py:128-145
  • Root Cause: _load_pddrc_config only validates that the root is a dict with a contexts key, then returns the config unchanged — no schema enforcement at root, context, or defaults nesting levels. Unknown keys are silently dropped, causing users to fall back to built-in defaults with no diagnostic output.
  • Test File: tests/test_construct_paths.py (class TestPddrcSchemaValidation + 3 standalone functions)

Fix Required In

  1. pdd/construct_paths.py — add _PDDRC_ROOT_KEYS, _PDDRC_CONTEXT_KEYS, _PDDRC_DEFAULTS_KEYS constants and a validator function; call it from _load_pddrc_config
  2. pdd/templates/generic/generate_pddrc_YAML.prompt — remove 9 occurrences of auto_deps_csv_path
  3. examples/prompts_linter/.pddrc — remove auto_deps_csv_path from all 5 contexts
  4. examples/prompts_linter/README.md — remove auto_deps_csv_path from YAML snippet

E2E Classification

E2E_NEEDED: no — test_construct_paths_triggers_validation_via_load exercises the full construct_paths → _load_pddrc_config → validator call chain with only the unrelated output-path generation mocked; Python's warnings.warn() propagates to CLI users without any CLI-specific wiring.

Next Steps

  1. Fix the bug at the identified location
  2. Run the test to confirm the fix
  3. Run full test suite to check for regressions
  4. Submit PR with fix and test

E2E skipped — proceeding to Step 12: Create Draft PR

@prompt-driven-github prompt-driven-github Bot mentioned this pull request May 27, 2026
7 tasks
@prompt-driven-github

Copy link
Copy Markdown
Contributor

Step 12: Draft PR Created

Pull Request

PR #1265: test: add failing tests for #1217 (.pddrc schema validation)

Branch

fix/issue-1217

What's Included

  • 12 failing tests at tests/test_construct_paths.py
    • 9 schema-validation tests in TestPddrcSchemaValidation
    • 3 data-file tests verifying removal of stale auto_deps_csv_path from example files
  • Commits: 1

Tests That Fail on Current Code

Test Failure Reason
test_unknown_root_key_emits_warning _load_pddrc_config emits no warning for unknown root keys
test_unknown_context_key_emits_warning _load_pddrc_config emits no warning for unknown context keys
test_unknown_defaults_key_emits_warning _load_pddrc_config emits no warning for unknown defaults keys
test_clean_config_emits_no_warnings (will pass when validator is implemented correctly)
test_warning_message_format No warnings emitted; format cannot be verified
test_auto_deps_csv_path_warns No warning emitted for stale auto_deps_csv_path key
test_prompt_path_warns No warning emitted for undocumented prompt_path key
test_construct_paths_triggers_validation_via_load E2E chain emits no warnings
test_validator_warns_on_multiple_unknown_keys No warnings emitted for any unknown key
test_examples_prompts_linter_pddrc_no_auto_deps_csv_path examples/prompts_linter/.pddrc still contains auto_deps_csv_path
test_generate_pddrc_yaml_template_no_auto_deps_csv_path_in_example_yaml Template still prescribes auto_deps_csv_path
test_examples_prompts_linter_readme_no_auto_deps_csv_path_in_yaml_snippets README still documents auto_deps_csv_path

Next Steps for Maintainers

  1. Review the failing tests to understand the expected behavior
  2. Implement the fix in pdd/construct_paths.py (lines 128–145):
    • Add _PDDRC_ROOT_KEYS, _PDDRC_CONTEXT_KEYS, _PDDRC_DEFAULTS_KEYS schema constants
    • Add a validator function emitting UserWarning per unknown key
    • Call validator from _load_pddrc_config
  3. Remove stale auto_deps_csv_path from template, example .pddrc, and example README
  4. Verify all 12 tests pass
  5. Run full test suite to check for regressions
  6. Mark the PR as ready for review

PDD Fix Command

To auto-fix this bug using PDD:

pdd fix https://github.com/promptdriven/pdd/issues/1217

Tip: Use --protect-tests if the tests are known to be correct, or --max-cycles N to limit fix attempts.


Investigation complete. A draft PR with failing tests has been created and linked to this issue.

prompt-driven-github Bot pushed a commit that referenced this pull request May 27, 2026
@prompt-driven-github

Copy link
Copy Markdown
Contributor

PDD Execution Successful

Command: bug
Duration: 45.2 min
Cost: $0.4516 infra (LLM billed to your API keys)
Pull Request: #1265

@gltanaka gltanaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous artifact blockers are mostly cleaned up now: pdd-env/ and error_log.txt are no longer committed as files, git diff --check origin/main...HEAD is clean, CI is green, and the focused .pddrc validation tests pass locally.

I still would not merge this head because .gitignore still contains unrelated and potentially harmful deletions. The PR removes existing ignore/allowlist rules and comments such as:

  • pdd/_version.py, the setuptools-scm generated version file ignore
  • .pdd/meta/*.json, which changes the ignore behavior for generated .pdd/meta JSON files
  • several explicit .pdd/meta/... allowlist entries
  • the Issue #1006 run-report comment and run-report exceptions
  • .loop/

Those changes are not part of issue #1198 and could change what local/generated files are picked up by git. Adding ignores for the accidental local artifacts (pdd-env/, error_log.txt) is fine, but the existing .gitignore behavior should be preserved.

Required change before merge:

  1. Restore the unrelated .gitignore deletions and keep the PR scoped to the .pddrc validation/template/example/test changes. If you want to keep the new pdd-env/ and error_log.txt ignore entries, add them without removing the existing rules.

Local checks run on head 89828b0:

  • git diff --check origin/main...HEAD -> clean
  • python -m pytest tests/test_construct_paths.py -q -> 114 passed
  • python -m pytest tests/test_find_prompt_file.py::TestPddrcPrefixForNewModules::test_new_module_respects_pddrc_prompts_dir_prefix -q -> 1 passed
  • loading checked-in .pddrc files via _load_pddrc_config -> no warnings

@gltanaka gltanaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest update resolves my remaining blockers.

What I verified on head f548a79:

  • The committed pdd-env/ and error_log.txt artifacts are gone.
  • .gitignore now only appends ignores for pdd-env/ and error_log.txt; the existing .pdd/meta fingerprint/run-report rules are preserved.
  • git diff --check origin/main...HEAD is clean.
  • python -m pytest tests/test_construct_paths.py -q -> 120 passed.
  • python -m pytest tests/test_find_prompt_file.py::TestPddrcPrefixForNewModules::test_new_module_respects_pddrc_prompts_dir_prefix -q -> 1 passed.
  • Loading checked-in .pddrc files through _load_pddrc_config produced no unknown-key warnings.

The .pddrc warning behavior looks scoped and useful now, and the previous PR hygiene issues are fixed. I think this is reasonable to merge once branch protection is satisfied.

@gltanaka gltanaka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approved after updating the branch with main. Verified updated head 594f702 locally: diff remains scoped, git diff --check is clean, tests/test_construct_paths.py passes, and the targeted prompt-dir regression passes.

@gltanaka
gltanaka merged commit 19e4150 into promptdriven:main May 28, 2026
5 checks passed
sohni-tagirisa added a commit that referenced this pull request May 28, 2026
Address PR review feedback from @gltanaka:
- Revert pdd/construct_paths.py to main (whitespace-only churn)
- Revert tests/test_agentic_sync.py to main (assertion-order swap)
- Revert tests/test_construct_paths.py to main (duplicate #1198 tests
  added by bot fix-loop; the canonical tests are already on main via #1217)

Keeps the actual fix-loop circuit breaker change and its tests intact.
@sohni-tagirisa sohni-tagirisa mentioned this pull request May 28, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.pddrc silently ignores unknown keys (environmental variables); no schema validation

2 participants