feat: use pattern-based pytest.raises(match=) in test-generation prompts (#1231) - #1235
Merged
Conversation
…pts (#1231) Update generate_test_LLM.prompt, generate_test_from_example_LLM.prompt, and context/test.prompt to require regex patterns (match=r"keyword1|keyword2") or type-only assertions instead of exact-string match= arguments, preventing test failures when error messages are reworded during regeneration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This was referenced May 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Updates the test-generation prompts to require regex pattern matching (
match=r"keyword1|keyword2") or type-only assertions inpytest.raises()calls instead of exact-string literals. This prevents cascading test failures when LLM regeneration rewords error messages while preserving exception type and behavior.Closes #1231
Changes Made
Prompts Modified
pdd/prompts/generate_test_LLM.prompt— Added CRITICAL rule prohibiting exact-stringmatch=arguments; requiresmatch=r"keyword1|keyword2"patterns or type-only assertionspdd/prompts/generate_test_from_example_LLM.prompt— Added the same CRITICAL rule before thecontext/test.promptincludeContext Updated
context/test.prompt— Updated SUMMARY OF GOOD PATTERNS bullet to mention pattern-based matching; added new Section 15: PYTEST.RAISES MATCH PATTERNS with explicit BAD/GOOD examplesSupporting Files
docs/prompting_guide.md— Minor update to align with pattern-matching guidancearchitecture.json— Updated to reflect prompt changesdocs/prompting_guide.md— diverged older mirror not in include graph — review contract-aware test generation block (item 6)Review Checklist
Next Steps After Merge
Created by pdd change workflow