Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show code coverage in report #2662

Open
ISErikMeijer opened this issue Aug 30, 2023 · 6 comments
Open

Show code coverage in report #2662

ISErikMeijer opened this issue Aug 30, 2023 · 6 comments
Labels
🚀 Feature request New feature or request hacktoberfest https://hacktoberfest.digitalocean.com

Comments

@ISErikMeijer
Copy link

If possible I would like if it showed the normal overall code coverage in the final stryker report, so I don't have to separately run a code coverage test.

@dupdob
Copy link
Member

dupdob commented Aug 30, 2023

That would be indeed beneficial.
Sadly, mutation coverage is very poor proxy for code coverage as it focuses on where Stryker injected mutations with no concern about non mutated code.
Furthermore, we capture coverage for a mutated version of the project, not the original one. As such:

  1. metrics would be wrong as Stryker adds to the code base
  2. coverage could be wrong (in subtle way) for some test/mutated code interferences.

In short, this is an interesting objective, but I do not see this happening in a near or medium future

@dupdob
Copy link
Member

dupdob commented Aug 30, 2023

that being said, the mutation coverage data can be published. But, again, this is a different report.

@richardwerkman
Copy link
Member

@dupdob
Copy link
Member

dupdob commented Aug 30, 2023

We could capture code coverage as part of the initial test run (that could be used as a normal test run BTW), but no use benefit beyond that. I am probably missing something important, but:

  1. As pointed above, code coverage results for mutated code may not reflect coverage for normal code
  2. Coverlet does not capture coverage per test, only per session. This would significantly impair coverage based optimization, as we could only determine which mutants are tested, without knowing which test(s) cover(s) them

There are also significant hurdle to address

  1. Ensure assembly PDBs contain location information for mutation (as of today, mutated code bears no location information)
  2. Parse coverage result to determine mutant coverage

@richardwerkman
Copy link
Member

@dupdob I don't think we need to use the coverage information to determine mutant coverage. We can keep that like it is. We just need to add coverage analysis to the initial testrun and get the percentage from it. The initial testrun doesn't contain mutations so the coverage should be correct.

@dupdob
Copy link
Member

dupdob commented Aug 31, 2023

oh, ok. I feel better then :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🚀 Feature request New feature or request hacktoberfest https://hacktoberfest.digitalocean.com
Projects
Status: No status
Development

No branches or pull requests

3 participants