Skip to content

[Feature]: Add CodeScene code coverage gates for pull requests #157

@stiwicourage

Description

@stiwicourage

Summary

Add CodeScene Code Coverage Gates for pull requests in NovaModuleTools by building on the repository’s existing Cobertura coverage generation and CodeScene upload flow.

Problem or use case

NovaModuleTools already does most of the hard work needed for CodeScene coverage gates:

  • .github/workflows/Tests.yml runs ./scripts/build/ci/Invoke-NovaModuleToolsCI.ps1 -OutputDirectory ./artifacts
  • that script generates artifacts/pester-coverage.cobertura.xml
  • the script remaps coverage back to source paths before writing the final Cobertura file
  • the workflow uploads artifacts/ as a GitHub artifact
  • the workflow already has a codescene-analysis job that installs cs-coverage, uploads coverage, and can trigger CodeScene analysis for selected events

However, this is still not the same as enabling CodeScene PR coverage gates.

The missing part is the PR-oriented cs-coverage check job that downloads coverage artifacts and posts/enforces the coverage-gate status check in GitHub pull requests.

Proposed solution

Implement CodeScene Code Coverage Gates in NovaModuleTools using the current CI setup as the base.

Concretely, the feature should include:

  1. Add a reusable GitHub Action based on the CodeScene template, for example:

    • .github/actions/check-coverage/action.yml
  2. Extend .github/workflows/Tests.yml with a dedicated PR coverage-gate job that:

    • depends on test-and-coverage
    • checks out with fetch-depth: 0
    • downloads the uploaded coverage artifacts
    • runs the CodeScene cs-coverage check command
  3. Reuse the existing Cobertura output that the repository already produces:

    • artifacts/pester-coverage.cobertura.xml
    • a matching glob would likely be **/pester-coverage.cobertura.xml
  4. Provide the CodeScene project endpoint to the gate job, for example by composing it from the existing values:

    • CS_URL
    • CS_PROJECT_ID
    • or a direct CS_PROJECT_URL
  5. Keep the current codescene-analysis job if it is still useful for coverage upload and manual/develop analysis runs, but separate that from the PR gate check.

  6. Enable CodeScene’s Code Coverage Gates in the CodeScene project configuration once the workflow exists.

    • repository badges indicate the CodeScene project is 78904
    • the most useful first gate is “New and Changed Code”
    • “Overall Code Coverage” can be added if the team wants an overall threshold too

Alternatives considered

The repository could continue with the current Codecov plus CodeScene upload/analysis flow, but that still does not provide the PR status check behavior described in CodeScene’s coverage-gate guide.

Another option would be to extend Invoke-CodeSceneAnalysis.ps1 so it performs the PR gate check too, but using the documented GitHub Action template from CodeScene seems closer to their intended integration model and easier to reason about in pull request workflows.

Affected area

Build or test flow

Related command or cmdlet

.github/workflows/Tests.yml, scripts/build/ci/Invoke-NovaModuleToolsCI.ps1, scripts/build/ci/Invoke-CodeSceneAnalysis.ps1

Example usage

# Existing CI already produces:
artifacts/pester-coverage.cobertura.xml

# New PR job should download that artifact and run a CodeScene
# coverage gate check as part of the pull request workflow.

Relevant project.json context

{
  "SetSourcePath": true
}

Expected benefit

Better automation or CI/CD

Breaking change risk

Low

Current NovaModuleTools version

2.1.1-preview04

Additional context

Reference: https://codescene.io/docs/guides/code-coverage-gates/check-code-coverage-in-pull-and-merge-requests.html

Current repository findings:

  • Tests.yml already uploads artifacts and sends coverage to Codecov
  • Invoke-NovaModuleToolsCI.ps1 already emits Cobertura coverage and remaps it back to source paths for CodeScene
  • Invoke-CodeSceneAnalysis.ps1 already supports cs-coverage upload and triggering analysis
  • what is missing is the PR-time cs-coverage check workflow integration that posts / enforces CodeScene coverage gates in GitHub

Pre-submit checklist

  • I searched existing issues to avoid creating a duplicate.
  • I explained the problem or use case, not just the desired solution.
  • This is not a bug report or security vulnerability report.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions