Skip to content

fix(workflows): close Codecov integration gaps for Pester and pytest flags#1106

Merged
WilliamBerryiii merged 3 commits intomainfrom
fix/codecov-integration-gaps
Mar 19, 2026
Merged

fix(workflows): close Codecov integration gaps for Pester and pytest flags#1106
WilliamBerryiii merged 3 commits intomainfrom
fix/codecov-integration-gaps

Conversation

@WilliamBerryiii
Copy link
Member

@WilliamBerryiii WilliamBerryiii commented Mar 17, 2026

Description

This PR closes four Codecov integration gaps identified in #1103 and addresses the root cause behind missing Codecov comments on fork PRs like #1095.

The Pester test runner (pester.config.ps1) dynamically discovers skill scripts under .github/skills/**/scripts/, but the Codecov flag mapping only listed scripts/**. Combined with require_changes: true, Codecov silently suppressed PR comments when changed files fell outside the mapped paths. This PR expands the pester flag paths, removes the require_changes gate, adds per-flag project and patch status targets, fixes a pytest coverage source mismatch, and establishes weekly coverage baselines.

Codecov Configuration

Root cause: PRs touching only skill scripts received no Codecov comment because the pester flag paths didn't include .github/skills/**/scripts/** and require_changes: true suppressed comments when no flag-relevant changes were detected.

  • Expanded pester flag paths to include .github/skills/**/scripts/**, matching the test runner's dynamic discovery scope
  • Removed require_changes: true from the comment section so fork PRs always receive Codecov feedback
  • Added pester project status target (auto with 1% threshold) and flag binding, matching the existing pytest project status
  • Added flag-specific patch coverage for both pester and pytest (80% target, informational: true)
    • Previously only the default patch target existed, so per-flag patch regressions went unreported

Pytest Workflow

  • Changed --cov=. to --cov in pytest-tests.yml so coverage honors each skill's pyproject.toml [tool.coverage.run] source setting rather than measuring from the working directory root

Weekly Validation

Weekly coverage baselines ensure Codecov has recent data for carryforward flag comparisons.

  • Added a Pester Coverage Baseline job invoking pester-tests.yml with soft-fail: true, changed-files-only: false, and code-coverage: true
  • Added a Discover Python Projects job that finds all pyproject.toml files and outputs a JSON matrix of project directories
  • Added a Pytest Coverage Baseline matrix job invoking pytest-tests.yml per discovered project, establishing weekly baselines

Related Issue(s)

Closes #1103

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

  • Verified the diff contains only intended changes (caught and reverted an unintended setup-uv SHA downgrade during review)
  • Confirmed actions/checkout in the new weekly discovery job uses the same pinned SHA (v4.2.2) with persist-credentials: false
  • All new weekly jobs use minimal permissions (contents: read + id-token: write for Codecov OIDC)

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps

@WilliamBerryiii
fix(workflows): close Codecov integration gaps for Pester and pytest …
711b2d4 
…flags

- add pester project status and flag-specific patch coverage to codecov.yml
- add skill script paths to pester flag so coverage-relevant PRs get comments
- remove require_changes so fork PRs always receive Codecov comments
- change pytest-tests.yml --cov=. to --cov to honor pyproject.toml source
- add weekly pester and pytest coverage baseline jobs to weekly-validation.yml

Closes #1103

📊 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii requested a review from a team as a code owner March 17, 2026 05:33
@github-actions
Copy link
Contributor

github-actions bot commented Mar 17, 2026 •
edited
Loading

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
actions/actions/checkout de0fac2e4500dabe0009e67214ff5f5447ce83dd 🟢 5.9
Details
CheckScoreReason
Maintained⚠️ 23 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 2
Code-Review🟢 10all changesets reviewed
Binary-Artifacts🟢 10no binaries found in the repo
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Packaging⚠️ -1packaging workflow not detected
Fuzzing⚠️ 0project is not fuzzed
License🟢 10license file detected
Pinned-Dependencies🟢 3dependency not pinned by hash detected -- score normalized to 3
Signed-Releases⚠️ -1no releases found
Security-Policy🟢 9security policy file detected
Branch-Protection🟢 5branch protection is not maximal on development and all release branches
SAST🟢 8SAST tool detected but not run on all commits

Scanned Files

  • .github/workflows/weekly-validation.yml

@codecov-commenter
Copy link

codecov-commenter commented Mar 17, 2026 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.78%. Comparing base (3739963) to head (4425785).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1106      +/-   ##
==========================================
- Coverage   88.13%   86.78%   -1.36%     
==========================================
  Files          45       59      +14     
  Lines        7932     8687     +755     
==========================================
+ Hits         6991     7539     +548     
- Misses        941     1148     +207
Flag Coverage Δ
pester 85.34% <ø> (-1.58%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 17 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@katriendg katriendg mentioned this pull request Mar 17, 2026
Closed
katriendg
katriendg approved these changes Mar 18, 2026
View reviewed changes
Copy link
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks

@WilliamBerryiii WilliamBerryiii merged commit cca29b7 into main Mar 19, 2026
30 checks passed
This was referenced Mar 19, 2026
Closed
Merged
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
WilliamBerryiii added a commit that referenced this pull request Mar 20, 2026
@hve-core-release-please @WilliamBerryiii
chore(main): release hve-core 3.2.0 (#1050)
c5cca83 
🤖 I have created a release *beep* *boop*
---


##
[3.2.0](hve-core-v3.1.46...hve-core-v3.2.0)
(2026-03-20)


### ✨ Features

* add -OutputPath parameter to Validate-MarkdownFrontmatter.ps1
([#1134](#1134))
([fdf1bcf](fdf1bcf)),
closes [#1006](#1006)
* add action version consistency scan workflow
([#1127](#1127))
([4229df1](4229df1))
* **agent:** MVE Experiment Designer
([#976](#976))
([70f86ca](70f86ca))
* **agents:** add ADO Backlog Manager orchestrator agent
([#800](#800))
([fae3987](fae3987))
* **agents:** add meeting analyst agent for transcript analysis using
work-iq ([#502](#502))
([5345b5b](5345b5b))
* **agents:** add quick-reference line to RPI Phase 5 suggestions
([#897](#897))
([9a90f39](9a90f39))
* **agents:** add RAI Planner, enhance SSSC Planner, and redesign
Security Planner
([#979](#979))
([06f826c](06f826c))
* **agents:** add symmetric cross-system handoff to GitHub Backlog
Manager ([#952](#952))
([ba34a35](ba34a35))
* **agents:** Functional Code Review Agent — pre-PR functional
correctness reviewer
([#733](#733))
([9cf63b7](9cf63b7))
* **build:** add Python extensions and uv 0.10.8 to devcontainer
([#920](#920))
([9ca0579](9ca0579))
* **build:** add uv ecosystem to Dependabot configuration
([#913](#913))
([2a4bd39](2a4bd39))
* **build:** enable npm pinning enforcement in dependency scan
([#838](#838))
([4e9e31f](4e9e31f))
* **build:** migrate attestation actions to v4.1.0 and add SBOM
verification docs
([#841](#841))
([ca1e65b](ca1e65b))
* **collections:** add four new validator checks (orphan, duplicate,
companion, coverage)
([#869](#869))
([1a96b73](1a96b73))
* **devcontainer,security:** add enterprise artifact hub configuration
([#1032](#1032))
([1d56d25](1d56d25))
* **docs:** add Rust coding standards and guidelines
([#809](#809))
([d4c4899](d4c4899))
* **extension:** add Microsoft logo icon to VS Code Marketplace listings
([#906](#906))
([82aca41](82aca41))
* **github:** add declarative label management
([#953](#953))
([a1a6845](a1a6845))
* **instructions:** add ADO backlog shared infrastructure
([#786](#786))
([1914078](1914078))
* **instructions:** add ADO backlog sprint planning and capacity
tracking ([#788](#788))
([d6fb77d](d6fb77d))
* **instructions:** add ADO triage workflow and prompt
([#787](#787))
([cde0190](cde0190))
* **instructions:** add shared story quality conventions and sprint
planning ([#803](#803))
([a2f18e3](a2f18e3))
* **prompts:** add ADO discovery and work item prompts with agent
routing ([#790](#790))
([7e74523](7e74523))
* **prompts:** add security review prompts
([#1118](#1118))
([ad30967](ad30967))
* **scripts:** add dynamic Python skill discovery for lint/test
([#957](#957))
([0a90f57](0a90f57))
* **scripts:** add Get-StandardTimestamp utility to CIHelpers module
([#1126](#1126))
([b273a4b](b273a4b))
* **scripts:** add Python copyright header validation
([#905](#905))
([67df902](67df902))
* **scripts:** add Python skill support to Validate-SkillStructure
([#903](#903))
([68479d9](68479d9))
* **scripts:** add workflow npm command scanning to dependency pinning
([#837](#837))
([6b5ae06](6b5ae06))
* **security:** add basic security reviewer agent with owasp skills
([#1008](#1008))
([cb1fd05](cb1fd05))
* **security:** add sigstore attestation bundles and fix
component-detection action
([#1148](#1148))
([f79c272](f79c272))
* **skills:** add Atheris fuzz harness with CI workflow integration
([#1102](#1102))
([d337e1d](d337e1d))
* **skills:** add PowerPoint automation skill with YAML-driven deck
generation ([#868](#868))
([00465cd](00465cd))
* **skills:** convert hve-core-installer agent to self-contained skill
([#846](#846))
([1d821fb](1d821fb))
* **skills:** enhance pr-reference skill with flexible filtering and
base branch detection
([#1095](#1095))
([26a32ea](26a32ea))
* **workflows:** add devcontainer infrastructure change log workflow
([#899](#899))
([8aca446](8aca446))
* **workflows:** add milestone auto-close on stable and pre-release
publishes ([#834](#834))
([79362b1](79362b1))
* **workflows:** add ms.date documentation freshness checking
([#969](#969))
([3ed441c](3ed441c))
* **workflows:** add Python linting CI workflow with Ruff
([#951](#951))
([f89f0eb](f89f0eb))
* **workflows:** add Python testing CI workflow with pytest and Codecov
([#934](#934))
([5e8306f](5e8306f))
* **workflows:** add uv and Python package sync to copilot-setup-steps
([#921](#921))
([45d517d](45d517d))


### 🐛 Bug Fixes

* **build:** override Linguist vendored flag for Python skill files
([#1155](#1155))
([0eee5b6](0eee5b6))
* **build:** override serialize-javascript to &gt;=7.0.3 for RCE fix
([#876](#876))
([e49039a](e49039a))
* **build:** resolve Pinned-Dependencies alerts for vsce npm commands in
extension workflows
([#782](#782))
([89dad9d](89dad9d))
* **build:** update undici and yauzl overrides for security audit
([#1030](#1030))
([2c2f92f](2c2f92f))
* **docs:** add CLI Plugins to install.md navigation surfaces
([#902](#902))
([79d6595](79d6595))
* **docs:** add sidebar ordering for Design Thinking documentation
([#832](#832))
([551fddc](551fddc)),
closes [#830](#830)
* **docs:** graduate design-thinking to preview and correct stale
collection references
([#831](#831))
([5110e35](5110e35))
* **docs:** include project-planning in UX Designer install guidance
([#908](#908))
([e7aa9bc](e7aa9bc))
* **docs:** remediate writing-style convention violations
([#865](#865))
([68b04bc](68b04bc))
* **docs:** remove draft content announcement banner
([#825](#825))
([b45de80](b45de80))
* **docs:** remove unbounded path-to-regexp override breaking SSG
([#1153](#1153))
([d810018](d810018))
* **docs:** use actual clone paths instead of folder display names in
multi-root workspace settings
([#984](#984))
([5dbab82](5dbab82))
* **instructions:** replace black with ruff in uv-projects
([#898](#898))
([b0c06d9](b0c06d9))
* **scripts:** cover .github/ skill files in copyright header validation
([#1055](#1055))
([#1098](#1098))
([27fbd33](27fbd33))
* **scripts:** eliminate phantom git changes from plugin generation
([#1035](#1035))
([e49a1b5](e49a1b5))
* **scripts:** enable JSON log output for lint:version-consistency
([#1033](#1033))
([52b0885](52b0885))
* **security:** calculate compliance score from total scanned
dependencies ([#930](#930))
([c112c3d](c112c3d))
* **skills:** add AST validation and namespace restriction for
content-extra.py
([#1027](#1027))
([c50c7a3](c50c7a3))
* **skills:** add depth limits to recursive PowerPoint processing
functions ([#1028](#1028))
([bf08994](bf08994))
* **skills:** harden XML parsing and blob writes in powerpoint extract
([#1053](#1053))
([89d24b1](89d24b1))
* **skills:** resolve ruff lint and format violations in powerpoint
skill ([#1048](#1048))
([17bbe7a](17bbe7a))
* **workflows:** add uv.lock dependencies submission have fork-skip
condition ([#1109](#1109))
([dec56ac](dec56ac))
* **workflows:** automate weekly SHA staleness check with issue creation
([#975](#975))
([1ea4caa](1ea4caa))
* **workflows:** close Codecov integration gaps for Pester and pytest
flags ([#1106](#1106))
([cca29b7](cca29b7))
* **workflows:** propagate uv sync errors in copilot-setup-steps
([#961](#961))
([df88d7c](df88d7c))
* **workflows:** resolve release-please skip cascade and Python project
discovery ([#1043](#1043))
([79993e2](79993e2))
* **workflows:** scan only commit subjects for breaking change detection
([#1157](#1157))
([a38a657](a38a657))


### 📚 Documentation

* clarify HVE Core Extension vs Installer messaging across documentation
([#965](#965))
([0fceb8f](0fceb8f))
* **docs:** add ADO integration user documentation
([#935](#935))
([ec89302](ec89302))
* **docs:** add Project Planning agent documentation
([#936](#936))
([3a3a0fd](3a3a0fd))
* **onboarding:** overhaul marketplace onboarding and documentation site
([#982](#982))
([4309e10](4309e10))


### ♻️ Refactoring

* **build:** merge code-review collection into coding-standards
([#863](#863))
([8027e7b](8027e7b))
* **workflows:** rename release pipeline workflows and add marketplace
automation triggers
([#829](#829))
([b6397f4](b6397f4))


### 🔧 Maintenance

* **build:** add clean:logs npm script
([#1122](#1122))
([f85fe02](f85fe02)),
closes [#988](#988)
* **build:** add JSON reporter for cspell
([#1123](#1123))
([6d59f67](6d59f67))
* **ci:** add multi-arch support to copilot-setup-steps binary downloads
([#955](#955))
([8d0c706](8d0c706))
* **deps-dev:** bump cspell from 9.6.4 to 9.7.0 in the npm-dependencies
group ([#839](#839))
([3fa16ff](3fa16ff))
* **deps:** bump actions/dependency-review-action from 4.8.3 to 4.9.0 in
the github-actions group across 1 directory
([#942](#942))
([1a9b858](1a9b858))
* **deps:** bump cairosvg from 2.8.2 to 2.9.0 in
/.github/skills/experimental/powerpoint
([#1025](#1025))
([f4deda7](f4deda7))
* **deps:** bump dompurify from 3.3.1 to 3.3.2 in /docs/docusaurus
([#924](#924))
([d2060d6](d2060d6))
* **deps:** bump svgo from 3.3.2 to 3.3.3 in /docs/docusaurus
([#880](#880))
([6dc2406](6dc2406))
* **deps:** bump the github-actions group across 1 directory with 4
updates ([#1100](#1100))
([2290dc0](2290dc0))
* **deps:** bump the github-actions group with 6 updates
([#840](#840))
([f57bc01](f57bc01))
* **docs:** correct New-MsDateReport table rendering and refresh stale
docs ([#1114](#1114))
([c2b806f](c2b806f))
* **settings:** remove orphaned Checkov config and stale gitignore
entries ([#870](#870))
([98fcd74](98fcd74))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: hve-core-release-please[bot] <254602402+hve-core-release-please[bot]@users.noreply.github.com>
Co-authored-by: Bill Berry <wberry@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@katriendg katriendg katriendg approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Codecov integration gaps: missing Pester status, pytest source mismatch, no per-flag patch coverage

3 participants

@WilliamBerryiii @codecov-commenter @katriendg