Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Setup codecov to block PRs from getting merged if they decrease coverage #139

Merged
merged 4 commits into from
Apr 5, 2023

Conversation

tnevrlka
Copy link
Contributor

https://issues.redhat.com/browse/STONEBLD-632

Implemented gating pull requests if they decrease overall test coverage by more than 1 %.
This threshold is subject to change based on feedback, but should cover most false negatives.
Behavior when removing covered lines is subject to change too. see possible alternatives

codecov/project check should be made mandatory by repo admins if lowering test coverage should actually block the ability to merge the PR.

Now ignoring pkg/github and pkg/gitlab because measuring their test coverage is not desired.

Also split Codecov upload into its own job, so it can be re-ran without re-running the tests. This job should fail in case of an error during upload.

Please see some examples of PRs:
decreased coverage
increased coverage, but overall decrease due to unexpected changes, still passed

See how test coverage can differ even between the same exact code:
coverage decreased even though it should not have changed, still passed
please compare with exact same code in a different PR, but with a different test coverage

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 16, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

tkdchen
tkdchen previously approved these changes Apr 3, 2023
Copy link
Contributor

@tkdchen tkdchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just notice this is still in draft.

@tkdchen tkdchen dismissed their stale review April 3, 2023 05:48

As the comment just added

@tnevrlka tnevrlka marked this pull request as ready for review April 4, 2023 07:21
@openshift-ci openshift-ci bot requested review from mmorhun and psturc April 4, 2023 07:22
Copy link
Collaborator

@mmorhun mmorhun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Let's try and see how it goes.

@psturc psturc merged commit 89ff42d into konflux-ci:main Apr 5, 2023
@tnevrlka tnevrlka deleted the code-coverage branch June 21, 2023 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants