Bug: pdd sync generated-test phase can pollute or break existing test files

[Serhan-Asad]

Ported from https://github.com/gltanaka/pdd/issues/1424

Problem

During global sync canaries, the generated-test phase appended large/noisy test suites directly into existing test files.

Observed cases:

• track_cost: generated tests produced a bad generated state with 8 failed, 58 passed, 25 errors and had to be manually removed.
• get_test_command: generated tests passed, but appended duplicate imports and local sys.path setup into tests/test_get_test_command.py.

Why this is genuine

This is not just stale prompt/code drift. Even after modules are synced, sync should not pollute existing tests or make test files noisier than necessary. The generated-test step needs stronger placement, isolation, and cleanup rules.

Expected behavior

• Generated tests should be isolated, focused, and idempotent.
• Existing checked-in tests should not receive duplicate imports, ad hoc sys.path setup, or broad generated dumps.
• If generated tests fail badly, sync should cleanly discard or quarantine them instead of leaving broken test files.

Evidence

Recorded in global_sync_investigation_notes.md under Track Cost Canary and Get Test Command Canary.

Acceptance criteria

• Add regression coverage for generated-test append behavior.
• Prevent duplicate imports/path setup when appending to existing tests.
• Quarantine or roll back generated tests when generated-test/fix produces large failure states.
• Ensure successful sync leaves a clean, reviewable test diff.

[Serhan-Asad]

5/8/2026

[Serhan-Asad]

I re-ran this from clean worktrees to check whether it is a genuine PDD test-generation bug.

Verification:

• Clean HEAD baseline for tests/test_track_cost.py: 20 passed.
• Clean pdd update output applied, then exact staging pdd sync --no-steer --budget 3 track_cost: sync failed earlier in generate on architecture conformance. It did not reach test generation, no tests were appended, and existing tests still passed.
• Clean worktree with the prompt + architecture state needed to get past generation: staging sync reached test, but PDD Cloud returned HTTP 500 before writing tests. Existing tests stayed unchanged and passed.
• Local fallback test generation on that same clean state did append tests, but they were valid: 66 passed, no duplicate test names, and only one import/header block.

Conclusion: I could not reproduce the reported bad generated-test state (8 failed, 58 passed, 25 errors) from a clean setup. The evidence points to the original observation coming from the dirty/manual canary state or a transient cloud/model run, not a deterministic PDD test append bug. Closing as not reproducible; we can reopen if there is a clean reproduction that leaves bad tests behind.