feat(skills): expand fuzz_has_formatting_variation to cover all formatting properties (#1143)#1296
Conversation
…iles - fix _has_formatting_variation dict-style branch to handle string colors - remove tracked .bin seed files and add corpus .gitignore - add dict-style test infrastructure to fuzz_harness - fix PS1 indentation and EOF newlines 🐛 - Generated by Copilot
- add *.bin pattern to root .gitignore for project-wide coverage - remove local corpus-specific .gitignore 🧹 - Generated by Copilot
katriendg
left a comment
There was a problem hiding this comment.
Thank you for this contribution. Expanding fuzz coverage for _has_formatting_variation is valuable work and aligns well with the deferred item from PR #1102. We appreciate the thoroughness of the new test cases covering all 6 formatting properties and the effort to add both object-style and dict-style test paths.
That said, we require some changes before this PR is ready to merge. Please review our PR Review comments.
…ormatting_variation - revert unrelated _has_formatting_variation refactor in extract_content.py - expand fuzz_harness.py with dict-style helper and pytest methods covering bold, italic, underline, size, and color_rgb without Mock classes - add SPDX header to generate_seeds.py and guard write_seed calls with __main__ - revert unrelated .gitignore *.bin entry 🛠️ - Generated by Copilot
|
Thanks for the thorough review @katriendg! Pushed 912b1bb addressing all five comments:
Local validation: |
|
Thanks for the thorough review @katriendg! Pushed 912b1bb addressing all five comments:
Local validation: |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1296 +/- ##
==========================================
- Coverage 87.64% 86.83% -0.82%
==========================================
Files 65 62 -3
Lines 10152 9462 -690
==========================================
- Hits 8898 8216 -682
+ Misses 1254 1246 -8
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
- hoist _Color helper out of _make_dict_run so isinstance comparisons succeed across calls (fixes test_identical_runs) - apply ruff format to fuzz_harness.py and generate_seeds.py 🐛 - Generated by Copilot
|
@katriendg ... this should be ready for a re-review ... I know we took a hit on our CodeCov ... but this does set us up nicely for some accessibility fuzzing down the road. |
katriendg
left a comment
There was a problem hiding this comment.
Thanks @WilliamBerryiii for applying the fixes, all good to merge now!
Description
Expanded the
fuzz_has_formatting_variationfuzz target to cover all formatting properties used by_has_formatting_variation.Previously, the fuzz harness only exercised:
font.namefont.boldfont.italicThis PR adds coverage for:
font.underlinefont.color.rgbfont.sizeAlso added corpus seed files under
tests/corpus/to improve fuzzing effectiveness and broaden input coverage.Related Issue(s)
Fixes #1143
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
prompt-builderagent and addressed all feedback.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md)Other:
.py)Testing
Ran:
All tests passing (28/28)
Verified coverage for:
Confirmed fuzz harness works in pytest mode and is compatible with Atheris fuzzing
Checklist
Required Checks
AI Artifact Contributions
/prompt-analyzeto review contributionprompt-builderreviewRequired Automated Checks
npm run lint:mdnpm run spell-checknpm run lint:frontmatternpm run validate:skillsnpm run lint:md-linksnpm run lint:psnpm run plugin:generateSecurity Considerations
Additional Notes