Skip to content
This repository has been archived by the owner on Nov 13, 2023. It is now read-only.

Random codecov diffs #461

Closed
mheuzey opened this issue Nov 11, 2020 · 1 comment
Closed

Random codecov diffs #461

mheuzey opened this issue Nov 11, 2020 · 1 comment
Labels
bug Something isn't working CI/CD priority-low Legit issue but cosmetic or nice-to-have severity-low Bug that makes the usage of the Zowe less convenient but doesn't impact key use cases testing Issues about testing.

Comments

@mheuzey
Copy link
Contributor

mheuzey commented Nov 11, 2020

When opening #458, I noticed that the codecov report initially showed some lines losing coverage in a file that I had not touched (CommandResponse.ts). The diff disappeared and reappeared as I made minor adjustments to the PR (no code changes). Looking into this topic of random codecov diffs, I found a thread which suggested that the tests might be non-deterministic. So I ran the unit test for that file on my local machine a few times, and it does look like the coverage changes between runs.

image

I'm not an expert on tests, but I figured I would bring this up so someone with more knowledge could verify whether something in the test suite could be causing random coverage changes. I don't know how widespread this problem is, but looking over the last few PRs, I see that #456 and #435 both have the same coverage diff as mine, and both PRs made small changes that shouldn't have affected coverage.

@zFernand0 zFernand0 added the testing Issues about testing. label Jul 27, 2021
@awharn awharn added CI/CD bug Something isn't working for-review To be reviewed in an Eng & Prod Mgmt meeting labels Dec 6, 2022
@zFernand0 zFernand0 added priority-low Legit issue but cosmetic or nice-to-have severity-low Bug that makes the usage of the Zowe less convenient but doesn't impact key use cases labels Dec 16, 2022
@JTonda
Copy link

JTonda commented Jan 30, 2023

We believe that updates to the codecov tools have eliminated this problem therefore we are closing this issue.

@JTonda JTonda closed this as completed Jan 30, 2023
@JTonda JTonda removed the for-review To be reviewed in an Eng & Prod Mgmt meeting label Jan 30, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working CI/CD priority-low Legit issue but cosmetic or nice-to-have severity-low Bug that makes the usage of the Zowe less convenient but doesn't impact key use cases testing Issues about testing.
Projects
None yet
Development

No branches or pull requests

4 participants