Skip to content

fix(security): calculate compliance score from total scanned dependencies#930

Merged
WilliamBerryiii merged 3 commits intomicrosoft:mainfrom
stablegenius49:pr-factory/issue-929-compliance-total-deps
Mar 8, 2026
Merged

fix(security): calculate compliance score from total scanned dependencies#930
WilliamBerryiii merged 3 commits intomicrosoft:mainfrom
stablegenius49:pr-factory/issue-929-compliance-total-deps

Conversation

@stablegenius49
Copy link
Contributor

@stablegenius49 stablegenius49 commented Mar 7, 2026

Summary

  • count total scanned dependencies per file type (GitHub Actions, npm, pip, shell downloads, workflow npm commands)
  • pass that total into Get-ComplianceReportData so compliance is based on scanned dependencies instead of violation count
  • remove the dead Severity -ne 'Info' filter from unpinned counting
  • add a unit test that verifies partial compliance (3 violations out of 10 dependencies => 70%)

Notes

  • keeps backward compatibility: if total dependency count is not supplied, the function falls back to violation count

Closes #929

Testing

  • Not run locally in this environment (PowerShell / Pester CLI not installed).

@stablegenius49 stablegenius49 requested a review from a team as a code owner March 7, 2026 12:27
@WilliamBerryiii
Copy link
Member

@stablegenius49 - please accept the CLA so that we can merge your PRs.

@stablegenius49
Copy link
Contributor Author

@microsoft-github-policy-service agree

…mpliance scoring

- scanner functions return @{ TotalCount; Violations } instead of flat arrays
- add mandatory TotalDependencies parameter to Get-ComplianceReportData
- add ValidateSet on DependencyViolation.Severity
- add CalculateScore() method to ComplianceReport class
- update all 118 Pester tests for new return format

Closes microsoft#929

🔧 - Generated by Copilot
@WilliamBerryiii
Copy link
Member

Hey @stablegenius49, thank you for jumping on this one! Your PR correctly identified the root cause — scanner functions weren't tracking total dependencies, making ComplianceScore binary — and your approach of adding counting functions was a solid way to solve it.

I happened to be working the same fix in parallel on #931, so I merged a complementary approach into your branch that refactors the scanner functions to return @{ TotalCount; Violations } hashtables instead of flat arrays. This avoids the second pass through files and keeps the counting co-located with the scanning logic. I also added ValidateSet on DependencyViolation.Severity, a CalculateScore() method on ComplianceReport, and updated the full test suite (118 tests passing).

Your original commit is preserved in the branch history. Appreciate the contribution — looking forward to more!

@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 96.87500% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 85.88%. Comparing base (45d517d) to head (dfa3cf6).

Files with missing lines Patch % Lines
scripts/security/Test-DependencyPinning.ps1 96.87% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #930      +/-   ##
==========================================
+ Coverage   85.86%   85.88%   +0.01%     
==========================================
  Files          27       27              
  Lines        5152     5159       +7     
==========================================
+ Hits         4424     4431       +7     
  Misses        728      728              
Flag Coverage Δ
pester 85.88% <96.87%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/security/Modules/SecurityClasses.psm1 97.22% <ø> (ø)
scripts/security/Test-DependencyPinning.ps1 89.83% <96.87%> (+0.17%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@WilliamBerryiii WilliamBerryiii merged commit c112c3d into microsoft:main Mar 8, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ComplianceScore in Get-ComplianceReportData is always 0% or 100% — TotalDependencies counts violations, not scanned dependencies

4 participants