I like the idea of OpenCover. Since it's maintained by the guy who was last
maintaining PartCover and has now dropped it, it makes sense for us to
switch our coverage reporting to OpenCover.

For my teams, I've never been a believer in running coverage more than
weekly. It's important but too much focus on it leads to the creation of
tests that don't actually test anything. I'd certainly not want to see it
run every time we update a branch.

Initially, I think we should get it baked into our NUnit.proj file. To do
it on one of the CI services, we would need to get them to allow us
multiple jobs for the project, which seems like a good possibility.

Charlie

On Mon, Dec 15, 2014 at 10:43 AM, Rob Prouse notifications@github.com
wrote:

I just wanted to put this out there as an idea and possibly a future task.
I have been experimenting with OpenCover
https://github.com/OpenCover/opencover to run code coverage on the
project and used ReportGenerator
http://www.palmmedia.de/OpenSource/ReportGenerator to generate HTML
reports.

So far I've just run one of the framework tests, but we are getting 86.7%
coverage which is pretty good. It does highlight some areas that are poorly
tested, so I think it is a worthy exercise.

I am using the command line versions of both tools, but they are also
available as NuGet packages that add MSBuild tasks.

An alternative might also be to use JetBrains' DotCover. They have a free
license for open source teams that we could take advantage of.

—
Reply to this email directly or view it on GitHub
#423.

I have also noticed that the open source version of ncover has had some development activity by other folks since the original developer formed his company.

Since this is clearly something we should do at some point, I'm taking it out of the 'idea' category. I recognize it may not actually get done for 3.0.

Interesting. Where have you seen this activity?

On 18 July 2015 at 16:43, CharliePoole notifications@github.com wrote:

I have also noticed that the open source version of ncover has had some
development activity by other folks since the original developer formed his
company.

Since this is clearly something we should do at some point, I'm taking it
out of the 'idea' category. I recognize it may not actually get done for
3.0.

—
Reply to this email directly or view it on GitHub
#423 (comment).

Well, I can't confirm because sourceforge is offline. The odd thing is that I saw activity as late as 2013 but there is no binary release after 2004. I'll check again when I can.

Wow, Sourceforge has been down for quite a while. it looks like their
Subversion repository is down as well. They appear to be struggling!

On 18 July 2015 at 22:33, CharliePoole notifications@github.com wrote:

Well, I can't confirm because sourceforge is offline. The odd thing is that I saw activity as late as 2013 but there is no binary release after 2004. I'll check again when I can.

—
Reply to this email directly or view it on GitHub.

@rprouse, I suggest your initial thoughts of using OpenCover and ReportGenerator are good. I have used these tools on two previous projects.

We added OpenCover and ReportGenerator to the Solution via NuGet and included the packages in source control so the CI server checked them out.

I have virtually generic batch files that can be executed as postbuild steps (we were using Jenkins)

The hardest part is to get the opencover exclusion syntax correct. It can be a bit tricky as the OpenCover documentation is not the best.

From duplicate #883

We should be encouraging best practices, so it would probably be a good idea to determine the code coverage of our tests and automatically publish it in our README.

Some ideas, use OpenCover or Mono Code Coverage (monocov) for coverage and use Coveralls.net to report coverage to Coveralls.io.

Made this a build task.

@dicko2 - I'm unassigning you from this issue for now - please feel free to grab it back if you want!

OpenCover and ReportGenerator nuget packages are always my never fail go to packages.

I also add OpenCoverToCoberturaConverter nuget package into the mix so I end up with test result XML that can be reported on (in my case) in Jenkins.

My batch process is:

1. OpenCover wrapped around NUnit
2. OpenCoverToCoberturaConverter
3. Report Generator

Then in post build I report on the NUnit Test results and publish HTML reports.

Works a charm. Has been my approach now for 5 years. Dependable!

Getting OpenCover filters correct the trickiest bit but after grappling with it for a while I can offer some insights if requested.

@agray : This is my first real run in with the .NET world and nunit. I have a use case which is the following.

Multiple NUnit dll's. Need to execute them all in one run. I can do this from powershell like this.

C:\tools\NUnit\3.8.0\nunit-console\nunit3-console.exe (ls -Recurse src\*\bin\*.Test.dll | % FullName) --result=NUnitResult.xml

Now I need to somehow wrap this for OpenCover? Would you have a suggestion for this?

@nunit/framework-team, would you be interested in using codecov.io? It adds and updates a comment like this one: OmerMor/AsyncBridge#16 (comment)

The codecov diff helped me catch this really subtle error. Would the PR comment be too noisy? We can also configure how much is in it or remove it altogether.

@jnm2 I would be for using codecov.io. I was planning to look into this or a similar code coverage tool for the analyzers project. And I don't mind the PR comments.

I used OpenCover at AsyncBridge since the tests are .NET Framework-only, but OpenCover works via debugger profiling API so it doesn't work with .NET Core yet.
Last night I set up AltCover on a project I test cross-platform. AltCover works with .NET Core because it IL-weaves instrumentation into the assemblies. (I chose AltCover rather than Coverlet because from what I see, Coverlet requires a NuGet reference in the csproj and it doesn't allow you to exclude the test assembly itself from the coverage results.)
If all goes well, I'll open a PR here.

I don't have a strong preference over framework, but I also like this idea. I found this after seeing a slack discussion from a bit ago about if we produced coverage reports. @OsirisTerje mentioned that the adapter is already doing this there and updating a badge in the readme to make the coverage visible.

Just updating this to bring us up to speed after it came up tangentially again.

I think the best way to proceed on this is going to be to use Coverlet which has become very popular for this purpose. Coverlet also has a Cake Plugin which I'm thinking might integrate nicely with how we're currently doing things.

I may try to take this on, because why not? :)

@stevenaw Yes, you're right. The adapter uses it, and Coverlet is also included in the nunit template we maintain in the dotnet test templates repository. So we should absolutely use it for the rest.
About the readme, I added Shields.io badge to get the coverage displayed. Here is the link: https://github.com/nunit/nunit3-vs-adapter#readme