Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coverage report missing one of the projects #1130

Closed
Tyrrrz opened this issue Mar 22, 2021 · 6 comments
Closed

Coverage report missing one of the projects #1130

Tyrrrz opened this issue Mar 22, 2021 · 6 comments
Assignees
Labels
untriaged To be investigated

Comments

@Tyrrrz
Copy link

Tyrrrz commented Mar 22, 2021

Hello 👋🏻

I thoroughly scanned open issues but couldn't find an answer to my problem. It feels like I'm missing something obvious but can't figure out what. 😩

I have the following project structure:

MiniRazor.Compiler.Tests
|  references
+--------------> MiniRazor.Compiler
                     |  references
                     +---------------> MiniRazor.Runtime

When running dotnet test, I get the following report:

Passed!  - Failed:     0, Passed:    13, Skipped:     0, Total:    13, Duration: 7 s - MiniRazor.Compiler.Tests.dll (net5.0)

Calculating coverage result...
  Generating report 'MiniRazor\MiniRazor.Compiler.Tests\coverage.net5.0.opencover.xml'

+-------------------+--------+--------+--------+
| Module            | Line   | Branch | Method |
+-------------------+--------+--------+--------+
| MiniRazor.Runtime | 89,47% | 66,66% | 92,85% |
+-------------------+--------+--------+--------+

+---------+--------+--------+--------+
|         | Line   | Branch | Method |
+---------+--------+--------+--------+
| Total   | 89,47% | 66,66% | 92,85% |
+---------+--------+--------+--------+
| Average | 89,47% | 66,66% | 92,85% |
+---------+--------+--------+--------+

As you can see, for some reason, it only shows coverage for MiniRazor.Runtime, but not MiniRazor.Compiler. The projects are not special in any particular way.

Also, I was able to get Coverlet working perfectly well on numerous other projects, this is the only one that's giving me issues.

For reproduction purposes, here's the project: https://github.com/Tyrrrz/MiniRazor/tree/cc2c70bbbaf9ef43098fea40bb19993c6e454fd4 (note the commit hash, I've since made a workaround that fixes coverage, see next comment)
Just clone it and run dotnet test.
Here's the latest build log from CI: https://github.com/Tyrrrz/MiniRazor/runs/2170799134?check_suite_focus=true

Any help is greatly appreciated 🙏🏻

@Tyrrrz
Copy link
Author

Tyrrrz commented Mar 22, 2021

Right after I posted it I had an epiphany, decided to remove Fody from MiniRazor.Compiler and the coverage worked.

This is really weird, because I'm using Fody in other projects and it works just fine with coverage.

Do you have any ideas on what could be the issue?

@daveMueller daveMueller added the untriaged To be investigated label Mar 24, 2021
@daveMueller
Copy link
Collaborator

So I just took a look into this. Seems that Fody is somehow changing the embedded pdb. With Fody we can't read the debug data. System.Reflection.PortableExecutable.PEReader can't read the path of the pdb.

image

Without Fody it isn't a problem at all.

image

I played around with it a bit and it seems that changing debug type to full <DebugType>full</DebugType> helped if this is an option.

@Tyrrrz
Copy link
Author

Tyrrrz commented Mar 26, 2021

Thanks for looking into it @daveMueller!

For curiosity sake, here's another project that uses Fody and Coverlet, but the coverage is collected properly:
https://github.com/Tyrrrz/YoutubeExplode/tree/763d5edc7e0429331840694597b95fab8ebd18d8 (note some tests may be failing as of writing this, just remove them to get the coverage report). It also has <DebugType>embedded</DebugType>.

@daveMueller
Copy link
Collaborator

OK thanks I'll look into this again. Maybe coverlet really has an issue detecting embedded .pdb in some edge cases.

@daveMueller daveMueller self-assigned this Mar 27, 2021
@daveMueller
Copy link
Collaborator

For curiosity sake, here's another project that uses Fody and Coverlet, but the coverage is collected properly:
https://github.com/Tyrrrz/YoutubeExplode/tree/763d5edc7e0429331840694597b95fab8ebd18d8 (note some tests may be failing as of writing this, just remove them to get the coverage report). It also has <DebugType>embedded</DebugType>.

Has there really been coverlet coverage with a Fody reference? When I run the test there is the same issue with the assembly (no module detected).

image

Also when I check the last of your CI builds where all tests passed https://github.com/Tyrrrz/YoutubeExplode/runs/1811013194?check_suite_focus=true it was exactly the same.

image

@Tyrrrz
Copy link
Author

Tyrrrz commented Mar 29, 2021

Interesting. I had looked at the report uploaded to Codecov, but after more careful inspection, it seems it has not been updated since 28th of November. I had absolutely no idea that was the case, turns out I had stale coverage statistics for close to half a year lol.

I went back to the last commit that had actual coverage, which is 41123da. It's using:

  • Fody v6.3.0
  • ConfigureAwait.Fody v3.3.1
  • coverlet.msbuild v2.9.0

The coverage is collected correctly:

+----------------+-------+--------+--------+
| Module         | Line  | Branch | Method |
+----------------+-------+--------+--------+
| YoutubeExplode | 5,35% | 12,4%  | 5,36%  |
+----------------+-------+--------+--------+

+---------+-------+--------+--------+
|         | Line  | Branch | Method |
+---------+-------+--------+--------+
| Total   | 5,35% | 12,4%  | 5,36%  |
+---------+-------+--------+--------+
| Average | 5,35% | 12,4%  | 5,36%  |
+---------+-------+--------+--------+

However, there is no <DebugType>embedded</DebugType> property set on the project. If I add it, then it stops working.

The reason I used embedded was because of a recommendation found here. It seems that there even was some movement to make embedded the default value for DebugType: dotnet/sdk#2679

So to summarize:

  1. Sorry for misleading you!
  2. Fody and Coverlet do seem to work together, but only if <DebugType> is NOT set to embedded.

It seems this is a problem on Fody's side, as you have identified earlier in this thread, so I will close the issue. Thanks a lot :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
untriaged To be investigated
Projects
None yet
Development

No branches or pull requests

2 participants