Question about code coverage of test assemblies

[sungam3r]

Hi. I know that coverlet excludes test assemblies by default thanks to default value for IncludeTestAssembly setting. Nevertheless I've noticed that in some test reports for projects in our codebase there are still test assemblies like Company.CoolFeature.UnitTests. After investigation I found that this test assembly is referenced by another test assembly from repo. Why? Well, for example, because Company.CoolFeature.UnitTests shares some test models to use across test assemblies. So my questions are:

1. Is this behavior expected or a some sort of bug?
2. Does IncludeTestAssembly work only for "root" test assemblies and not for referenced ones?
3. What is the recommended way to 100% exclude test assemblies from our code coverage?

Thanks.

[Bertk]

Hi, there are some limitations e.q. generated code for test projects.

You can use /p:ExcludeByAttribute="GeneratedCodeAttribute" and reduce the noise. I use reportgenerator and generate coverage reports. This can be used to tweak the configuration for the coverage reports.

see also coverlet CI:

• build.yml
• publish-coverage-results.yml

<ExcludeByAttribute>Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute</ExcludeByAttribute>

[sungam3r]

I have nothing to do with GeneratedCodeAttribute. One test assembly references other test assembly - that is all.

[Bertk]

Please provide more details. In the past we had similar issues and some were created by Microsoft.Net.Test.Sdk code generation.
You should also check the filter option see ReportGenerator Command Line Parameter

[sungam3r]

I know about report generator and use it. The question is about coverlet behaviour, not reportgenerator. What details I can provide? I already said that situation is very simple - when one test project become reference other test project then I see coverage for referenced assembly even when it was excluded by filter like [*Tests]*. If I remove reference then filter works as expected.

[Bertk]

Sorry, but could you please explain the target element for the filter e.g. file names will not work.

example exclude filter used in coverlet CI

/p:Exclude="[coverlet.core.tests.samples.netstandard]*%2c[coverlet.tests.xunit.extensions]*%2c[coverlet.tests.projectsample]*%2c[testgen_]*"

[daveMueller]

@sungam3r I think this comment here explains it (#1503 (comment)).

1. Is this behavior expected or a some sort of bug?

Yes it is the expected behavior.

2. Does IncludeTestAssembly work only for "root" test assemblies and not for referenced ones?

Yes exactly. It only works for the test assembly being executed. If the root-(test)-assembly references other assemblies we can't really distinguish whether it is a test assembly or not.

3. What is the recommended way to 100% exclude test assemblies from our code coverage?

It may be best for your scenario to exclude the assembly with the test models by using filters. The other option would be to really separate the test assemblies in a way that they don't reference each other.

Have I been able to clarify all questions?

[sungam3r]

@daveMueller Thank you, this is exactly what I expected to hear. I had to make sure that coverlet works just like that.