Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorporate coverage diff reporting #1668

Closed
4 tasks
thekaveman opened this issue Aug 16, 2023 · 1 comment
Closed
4 tasks

Incorporate coverage diff reporting #1668

thekaveman opened this issue Aug 16, 2023 · 1 comment
Assignees
Labels
actions Related to GitHub Actions workflows chore Chores and tasks for code cleanup, dev experience, admin/configuration settings, etc. tests Related to automated testing (unit, UI, integration, etc.)

Comments

@thekaveman
Copy link
Member

thekaveman commented Aug 16, 2023

We use the coverage package (indirectly via pytest-cov) to calculate Python unit test coverage of the codebase.

The test runner generates a coverage report, the latest always available on our docs site at: https://docs.calitp.org/benefits/tests/coverage

However, within a given PR or set of changes we don't get much insight of the impact on test coverage -- without manually reviewing coverage reports and eyeballing differences.

The Python Coverage Comment GitHub Action can be used in a workflow to provide coverage diffs for our pytests. The action uses existing coverage metadata generated by the test runner to create a PR comment with information about the change. The comment is updated as the PR changes and new test runs are produced.

Example: comment on a first-time setup PR
image

Acceptance Criteria

  • Use coverage directly rather than via pytest-cov
  • Use pyproject.toml to configure coverage
  • Update pytest actions workflow to use Python Coverage Comment to provide coverage diff reporting, example

Additional context

Example of this setup

@thekaveman thekaveman added chore Chores and tasks for code cleanup, dev experience, admin/configuration settings, etc. tests Related to automated testing (unit, UI, integration, etc.) actions Related to GitHub Actions workflows labels Aug 16, 2023
@thekaveman thekaveman added this to the Reliability milestone Aug 16, 2023
@machikoyasuda machikoyasuda removed this from the Reliability [eng] milestone Aug 22, 2023
@machikoyasuda
Copy link
Member

This was completed with #1745 right?

@thekaveman thekaveman self-assigned this Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
actions Related to GitHub Actions workflows chore Chores and tasks for code cleanup, dev experience, admin/configuration settings, etc. tests Related to automated testing (unit, UI, integration, etc.)
Projects
Status: Done
Development

No branches or pull requests

2 participants