Skip to content

feat(skills): expand fuzz_has_formatting_variation to cover all formatting properties (#1143)#1296

Merged
WilliamBerryiii merged 12 commits intomicrosoft:mainfrom
PratikWayase:feat/fuzz-formatting-variation-1143
Apr 28, 2026
Merged

feat(skills): expand fuzz_has_formatting_variation to cover all formatting properties (#1143)#1296
WilliamBerryiii merged 12 commits intomicrosoft:mainfrom
PratikWayase:feat/fuzz-formatting-variation-1143

Conversation

@PratikWayase
Copy link
Copy Markdown
Contributor

Description

Expanded the fuzz_has_formatting_variation fuzz target to cover all formatting properties used by _has_formatting_variation.

Previously, the fuzz harness only exercised:

  • font.name
  • font.bold
  • font.italic

This PR adds coverage for:

  • font.underline
  • font.color.rgb
  • font.size

Also added corpus seed files under tests/corpus/ to improve fuzzing effectiveness and broaden input coverage.


Related Issue(s)

Fixes #1143


Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Other:

  • Script/automation (.py)
  • Other (please describe):

Testing

  • Ran:

    uv run pytest .github/skills/experimental/powerpoint/tests/fuzz_harness.py -v
  • All tests passing (28/28)

  • Verified coverage for:

    • underline variation
    • color.rgb variation
    • size variation
  • Confirmed fuzz harness works in pytest mode and is compatible with Atheris fuzzing


Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Plugin freshness: npm run plugin:generate

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes


@PratikWayase PratikWayase requested a review from a team as a code owner April 4, 2026 06:13
PratikWayase and others added 4 commits April 4, 2026 11:48
…iles

- fix _has_formatting_variation dict-style branch to handle string colors
- remove tracked .bin seed files and add corpus .gitignore
- add dict-style test infrastructure to fuzz_harness
- fix PS1 indentation and EOF newlines

🐛 - Generated by Copilot
- add *.bin pattern to root .gitignore for project-wide coverage
- remove local corpus-specific .gitignore

🧹 - Generated by Copilot
Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this contribution. Expanding fuzz coverage for _has_formatting_variation is valuable work and aligns well with the deferred item from PR #1102. We appreciate the thoroughness of the new test cases covering all 6 formatting properties and the effort to add both object-style and dict-style test paths.

That said, we require some changes before this PR is ready to merge. Please review our PR Review comments.

Comment thread .github/skills/experimental/powerpoint/scripts/extract_content.py
Comment thread .github/skills/experimental/powerpoint/tests/fuzz_harness.py
Comment thread .github/skills/experimental/powerpoint/tests/fuzz_harness.py Outdated
Comment thread .gitignore Outdated
…ormatting_variation

- revert unrelated _has_formatting_variation refactor in extract_content.py
- expand fuzz_harness.py with dict-style helper and pytest methods covering bold, italic, underline, size, and color_rgb without Mock classes
- add SPDX header to generate_seeds.py and guard write_seed calls with __main__
- revert unrelated .gitignore *.bin entry

🛠️ - Generated by Copilot
@WilliamBerryiii
Copy link
Copy Markdown
Member

Thanks for the thorough review @katriendg! Pushed 912b1bb addressing all five comments:

  1. extract_content.py — reverted the _has_formatting_variation refactor entirely. Original key-existence behavior preserved.
  2. fuzz_harness.py coverage — added a small _make_dict_run helper (plain dict + inline _Color) and explicit pytest methods for bold, italic, underline, size, and color_rgb variation. No Mock classes introduced.
  3. cairosvg side effect — removed the module-level sys.modules[cairosvg] = MagicMock() line.
  4. generate_seeds.py — added SPDX/copyright header and wrapped all write_seed calls in if __name__ == main: so importing the module is side-effect free.
  5. .gitignore — reverted the *.bin change.

Local validation: npm run validate:copyright, lint:py, and validate:skills pass.

@WilliamBerryiii
Copy link
Copy Markdown
Member

Thanks for the thorough review @katriendg! Pushed 912b1bb addressing all five comments:

  1. extract_content.py — reverted the _has_formatting_variation refactor entirely. Original key-existence behavior preserved.
  2. fuzz_harness.py coverage — added a small _make_dict_run helper (plain dict + inline _Color) and explicit pytest methods for bold, italic, underline, size, and color_rgb variation. No Mock classes introduced.
  3. cairosvg side effect — removed the module-level sys.modules["cairosvg"] = MagicMock() line.
  4. generate_seeds.py — added SPDX/copyright header and wrapped all write_seed calls in if __name__ == "__main__": so importing the module is side-effect free.
  5. .gitignore — reverted the *.bin change.

Local validation: npm run validate:copyright, lint:py, and validate:skills pass.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.83%. Comparing base (a9014c9) to head (efe0492).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1296      +/-   ##
==========================================
- Coverage   87.64%   86.83%   -0.82%     
==========================================
  Files          65       62       -3     
  Lines       10152     9462     -690     
==========================================
- Hits         8898     8216     -682     
+ Misses       1254     1246       -8     
Flag Coverage Δ
pester 85.02% <ø> (-0.02%) ⬇️
pytest 91.66% <ø> (-1.49%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- hoist _Color helper out of _make_dict_run so isinstance comparisons succeed across calls (fixes test_identical_runs)

- apply ruff format to fuzz_harness.py and generate_seeds.py

🐛 - Generated by Copilot
@WilliamBerryiii
Copy link
Copy Markdown
Member

@katriendg ... this should be ready for a re-review ... I know we took a hit on our CodeCov ... but this does set us up nicely for some accessibility fuzzing down the road.

Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @WilliamBerryiii for applying the fixes, all good to merge now!

@WilliamBerryiii WilliamBerryiii merged commit d51b4d3 into microsoft:main Apr 28, 2026
51 checks passed
@github-actions github-actions Bot mentioned this pull request Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(skills): expand fuzz_has_formatting_variation to cover all formatting properties

4 participants