Separate CI build and test concerns, add full codebase coverage report#1611
Merged
Conversation
- Build jobs now run only Rust tests with code coverage (removed redundant Pester 'resources' group that was duplicated by pester jobs) - Pester jobs now collect code coverage by setting LLVM_PROFILE_FILE and using llvm-profdata/llvm-cov to export LCOV from instrumented binaries - Coverage report job merges all LCOV files (Rust tests + Pester tests) instead of picking only the first file found - Added Export-PesterCodeCoverageReport helper function for generating LCOV from profraw files using llvm tools directly - Added Merge-LcovFile helper function to consolidate multiple LCOV reports into a single merged file Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Pester jobs now write profraw files to a unique temp directory instead of the workspace, with cleanup after LCOV generation - Coverage report now includes both changed-code coverage (existing) and full codebase coverage (new) in the PR comment - Added Get-FullCodeCoverageReport helper that computes overall line coverage across all instrumented source files Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR refactors the Rust CI workflow to avoid redundant Pester execution in build jobs, and enhances coverage reporting by incorporating coverage exercised by the Pester matrix jobs into a merged LCOV report (including a new full-codebase coverage section).
Changes:
- Build jobs now run Rust tests under coverage instrumentation without re-running Pester (
-ExcludePesterTests). - Pester matrix jobs now collect LLVM profraw data and export per-job LCOV artifacts.
- Coverage reporting now merges all LCOV inputs and posts both changed-code and full-codebase coverage summaries.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| helpers.build.psm1 | Adds helpers for exporting Pester-driven LCOV and merging/reporting coverage across jobs. |
| .github/workflows/rust.yml | Splits build vs. Pester responsibilities and aggregates coverage artifacts into a unified PR comment. |
- Fix Join-Path with multiple segments: use [System.IO.Path]::Combine() and validate host triple parse result - Add Test-Path check for llvm-cov (not just llvm-profdata) - Wrap Get-ChildItem for profraw files in @() with -ErrorAction SilentlyContinue to handle null/empty reliably - Fix Unix executable detection: use UnixMode property instead of shelling out to 'test -x' which returns no output in PowerShell - Wrap binaries result in @() to ensure array even when null/single - Remove unused SourceDirectory parameter and its documentation Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…cords - Parse DA hit counts as [decimal] and clamp values above [long]::MaxValue to 0 (matching Get-CodeCoverageReport behavior for LLVM sentinels) - Use portable Mode property instead of UnixMode for executable detection - Remove function-level LCOV records (FN/FNDA/FNF/FNH) from merged output since correct aggregation requires more than simple concatenation Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When many profraw files are produced (1600+), passing all paths as command-line arguments exceeds Windows' limit. Write the file list to a response file and pass it via the @file syntax instead. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Code Coverage ReportNo Rust files were changed in this PR. 🔴 Full Codebase Coverage48% (low)
|
Restructured the coverage report logic so the full codebase report is computed and displayed regardless of whether Rust files were modified in the PR. The changed-code section is still conditional on Rust changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
When running ./build.ps1 -CodeCoverage locally, the output now includes: - Full codebase coverage summary (always shown) - File-by-file breakdown with -CodeCoverageShowFiles - Uncovered line display with -CodeCoverageShowUncoveredLines - Configurable threshold and top-N file limits New helper functions: - Get-FullCodeCoverageDetail: parses LCOV into per-file coverage objects - Show-FullCodeCoverageReport: colorized file/line coverage display Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The CI workflow had two issues: the build jobs redundantly ran
resourcesPester tests thatthe pester matrix jobs also ran, and the pester jobs didn't contribute any code coverage data
to the coverage report.
Approach
Build/test separation: Build jobs now use
-ExcludePesterTestsso they only run Rusttests under
cargo-llvm-covinstrumentation (required for Rust code coverage). Pester testingis fully owned by the dedicated pester matrix jobs -- no more overlap.
Pester coverage collection: Each pester job now sets
LLVM_PROFILE_FILEpointing to aunique temp directory, so the instrumented binaries write profraw data when exercised by
integration tests. After tests complete, a new
Export-PesterCodeCoverageReporthelper usesllvm-profdata merge+llvm-cov exportto produce per-job LCOV files. Temp profraw datais cleaned up immediately after.
Merged coverage report: The coverage-report job now discovers and merges all LCOV files
(from Rust unit tests and from every Pester group on every platform) using a new
Merge-LcovFilehelper, instead of picking only the first file found.Full codebase coverage: The PR comment now includes two sections:
New helper functions in
helpers.build.psm1Export-PesterCodeCoverageReport-- generates LCOV from profraw files using llvm tools directlyMerge-LcovFile-- merges multiple LCOV files by summing line hit counts per source fileGet-FullCodeCoverageReport-- computes total line coverage across all source files in an LCOVNotes
rustup component add llvm-tools-preview(lightweight, ~10s) to accessthe llvm-profdata and llvm-cov binaries