|
| 1 | +# Test Case Enhancement Rules |
| 2 | + |
| 3 | +## Assistant Workflow for Adding Comprehensive Test Cases |
| 4 | + |
| 5 | +When user requests to enhance test cases for a problem, the assistant will: |
| 6 | + |
| 7 | +### 1. Problem Resolution (Priority Order) |
| 8 | + |
| 9 | +- **FIRST**: Try to resolve from context - check active file path or user-provided problem name |
| 10 | +- **SECOND**: If context resolution fails, THEN run `poetry run python .templates/check_test_cases.py --threshold=10 --max=1` to auto-detect 1 problem with <10 test cases |
| 11 | +- **LAST**: If both above fail, ask user to explicitly specify problem name |
| 12 | + |
| 13 | +### 2. Test Case Generation |
| 14 | + |
| 15 | +- Read `leetcode/{problem_name}/README.md` for problem understanding |
| 16 | +- Analyze existing test cases in `leetcode/{problem_name}/tests.py` |
| 17 | +- Generate comprehensive test cases covering: |
| 18 | + - **Edge cases**: Empty inputs, single elements, boundary values |
| 19 | + - **Corner cases**: Maximum/minimum constraints, special patterns |
| 20 | + - **Normal cases**: Typical scenarios with varied complexity |
| 21 | + - **Error cases**: Invalid inputs (if applicable) |
| 22 | + |
| 23 | +### 3. Initial Validation |
| 24 | + |
| 25 | +- Run `make p-test PROBLEM={problem_name}` to verify current implementation |
| 26 | +- **If errors found**: |
| 27 | + - DO NOT update implementation automatically |
| 28 | + - Only update test cases if they're incorrect |
| 29 | + - If implementation seems wrong, ASK USER first before modifying |
| 30 | + |
| 31 | +### 4. JSON Template Update |
| 32 | + |
| 33 | +- Update corresponding `.templates/leetcode/json/{problem_name}.json` |
| 34 | +- Add new test cases to `test_cases` field in proper format |
| 35 | +- Maintain existing test structure and naming conventions |
| 36 | + |
| 37 | +### 5. Backup and Regeneration Process |
| 38 | + |
| 39 | +- **Backup**: Move `leetcode/{problem_name}/` to `.cache/leetcode/{problem_name}/` |
| 40 | +- **Regenerate**: Run `make p-gen PROBLEM={problem_name} FORCE=1` |
| 41 | +- **Lint check**: Run `make p-lint PROBLEM={problem_name}` |
| 42 | +- **Iterate**: If lint fails, update JSON and regenerate until passes |
| 43 | + |
| 44 | +### 6. Solution Preservation |
| 45 | + |
| 46 | +- Copy `solution.py` from backup to newly generated structure |
| 47 | +- Run `make p-test PROBLEM={problem_name}` to verify tests pass |
| 48 | +- **If tests fail**: Go back to step 4, update JSON, and iterate until passes |
| 49 | + |
| 50 | +### 7. Cleanup and Restore |
| 51 | + |
| 52 | +- **CRITICAL**: Remove entire newly generated `leetcode/{problem_name}/` directory |
| 53 | +- **CRITICAL**: Restore original structure from `.cache/leetcode/{problem_name}/` backup |
| 54 | +- **CRITICAL**: Only THEN copy enhanced `test_solution.py` from generated files to restored structure |
| 55 | +- **CRITICAL**: Preserve existing solution class parametrization - if original test had multiple solution classes, restore them |
| 56 | +- Verify final state with `make p-test PROBLEM={problem_name}` |
| 57 | +- Clean up backup directory after successful verification |
| 58 | + |
| 59 | +## Test Case Quality Standards |
| 60 | + |
| 61 | +### Coverage Requirements |
| 62 | + |
| 63 | +- **Minimum 10 test cases** per problem |
| 64 | +- **Edge cases**: 20-30% of total test cases |
| 65 | +- **Normal cases**: 50-60% of total test cases |
| 66 | +- **Corner cases**: 20-30% of total test cases |
| 67 | + |
| 68 | +### Test Case Categories |
| 69 | + |
| 70 | +#### Edge Cases |
| 71 | + |
| 72 | +- Empty inputs: `[]`, `""`, `None` |
| 73 | +- Single element: `[1]`, `"a"` |
| 74 | +- Boundary values: `[0]`, `[1]`, `[-1]` |
| 75 | +- Maximum/minimum constraints from problem description |
| 76 | + |
| 77 | +#### Corner Cases |
| 78 | + |
| 79 | +- Duplicate elements: `[1,1,1]` |
| 80 | +- Sorted/reverse sorted arrays: `[1,2,3]`, `[3,2,1]` |
| 81 | +- All same elements: `[5,5,5,5]` |
| 82 | +- Alternating patterns: `[1,0,1,0]` |
| 83 | + |
| 84 | +#### Normal Cases |
| 85 | + |
| 86 | +- Mixed positive/negative numbers |
| 87 | +- Various array sizes within constraints |
| 88 | +- Different data patterns and structures |
| 89 | +- Representative problem scenarios |
| 90 | + |
| 91 | +### JSON Format Requirements |
| 92 | + |
| 93 | +- Use single quotes for Python strings in test cases |
| 94 | +- Follow existing parametrize format |
| 95 | +- Maintain type hints in parametrize_typed |
| 96 | +- Ensure test_cases string is valid Python list syntax |
| 97 | +- **NEVER include custom solution classes** in test_imports - only import the main solution class specified in solution_class_name |
| 98 | +- **PRESERVE existing solution class parametrization** - if original test had multiple solution classes, restore them after JSON regeneration |
| 99 | + |
| 100 | +## Commands Reference |
| 101 | + |
| 102 | +```bash |
| 103 | +# Find problems needing more test cases |
| 104 | +poetry run python .templates/check_test_cases.py --threshold=10 --max=1 |
| 105 | + |
| 106 | +# Test specific problem |
| 107 | +make p-test PROBLEM={problem_name} |
| 108 | + |
| 109 | +# Generate from JSON template |
| 110 | +make p-gen PROBLEM={problem_name} FORCE=1 |
| 111 | + |
| 112 | +# Lint specific problem |
| 113 | +make p-lint PROBLEM={problem_name} |
| 114 | +``` |
| 115 | + |
| 116 | +## Error Handling |
| 117 | + |
| 118 | +- **Implementation errors**: Ask user before modifying solution code |
| 119 | +- **Test failures**: Update JSON template and regenerate |
| 120 | +- **Lint failures**: Fix JSON format and iterate |
| 121 | +- **Backup failures**: Ensure `.cache/leetcode/` directory exists |
| 122 | + |
| 123 | +## Success Criteria |
| 124 | + |
| 125 | +- All tests pass with enhanced test cases |
| 126 | +- Minimum 10 comprehensive test cases per problem |
| 127 | +- Original solution code preserved and working |
| 128 | +- JSON template updated for future regeneration |
| 129 | +- Clean final state with no temporary files |
0 commit comments