Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish code coverage task overrides ReportGenerator filters #10354

Open
nilfdev opened this issue May 10, 2019 · 10 comments
Open

Publish code coverage task overrides ReportGenerator filters #10354

nilfdev opened this issue May 10, 2019 · 10 comments

Comments

@nilfdev
Copy link

@nilfdev nilfdev commented May 10, 2019

Required Information

Type: bug
Enter Task Name: PublishCodeCoverageResults@1

Environment

  • Server - Azure Pipelines
  • Agent - Private
  • dotnet-reportgenerator-cli Version="4.1.5"

Issue description

I am configuring azure CI pipeline. And I've added following steps:

  • test
  • report generator
  • publish code coverage

ReportGenerator is the well know tool https://github.com/danielpalme/ReportGenerator
that exists in the Azure Devops marketplace
image

in the ReportGenerator I apply filters to exclude test projects from the total test coverage report and exclude extra xUnit assemblies from analysis.

image

In the console log I see that following filters were applied:

-assemblyfilters:-[.tests] -[.Tests] -[xunit.]* -[.Test] -[Xunit.]*
-classfilters:-[.tests] -[.Tests] -[xunit.]* -[.Test] -[Xunit.]*
-filefilters:-[.tests] -[.Tests] -[xunit.]* -[.Test] -[Xunit.]*

as you also can see, on the next step Publishing code coverage task starts and it re-generates summary report one more time without applying the assembly filtering.
it became like this after integration with reportGenerator tool
https://docs.microsoft.com/en-us/azure/devops/release-notes/2019/sprint-150-update#cobertura-code-coverage-report-updates

how to apply filtering and exclude necessary files from the report on CI, while publishing code coverage step override the filter attributes. Can you also apply settings for filtering classes and assemblies

finale report has data that I don't really need
image

@ghost ghost added route triage labels May 10, 2019
@danielpalme

This comment has been minimized.

Copy link
Contributor

@danielpalme danielpalme commented May 12, 2019

Disclaimer: I'm the maintainer/author of ReportGenerator

I think the new behavior is not what users are expecting.
If you specify a "Report Directory" the users would expect to see exactly the specified report files to appear in the Code Coverage tab of the build results.
Only if the "Report Directory" is not supplied, the Publish Code Coverage Results task should generate a report by itself to enhance the user experience.

@nilfdev
To disable the regeneration of the report, you can use the following environment variable in your build:

disable.coverage.autogenerate: 'true'
@joshmgross joshmgross added Area: CodeCoverage and removed route labels May 13, 2019
@nilfdev

This comment has been minimized.

Copy link
Author

@nilfdev nilfdev commented May 14, 2019

@danielpalme yes.
disable.coverage.autogenerate fixes the problem.
it is not obvious setting.

@PBoraMSFT

This comment has been minimized.

Copy link
Member

@PBoraMSFT PBoraMSFT commented May 21, 2019

@nilfdev - thank you for reporting this issue. Adding first-class support for exclusion filters is on the backlog. The suggestion that @danielpalme has made can be used as a workaround in the meantime.

@cltshivash cltshivash added enhancement and removed triage labels Jun 20, 2019
@mnoreke

This comment has been minimized.

Copy link

@mnoreke mnoreke commented Jul 2, 2019

As I have struggled with this being new to Pipelines, I want to expand on @danielpalme 's suggestion. I tried passing it every which way as a parameter, but it needed to be set at the start of my Yaml file in the following block.

I have it just before the steps, and am listing that for someone else who might need a nudge in the right direction.

variables:
disable.coverage.autogenerate: 'true'

steps:

@Sharparam

This comment has been minimized.

Copy link

@Sharparam Sharparam commented Oct 17, 2019

The auto-generated reports don't seem to respect the current theme set in the DevOps GUI. Is this intended? There doesn't seem to be a way to configure the theme used by the auto-generated report, so is the only way to disable autogeneration using above methods, and keep generating it manually?

@danielpalme

This comment has been minimized.

Copy link
Contributor

@danielpalme danielpalme commented Oct 17, 2019

@Sharparam
An option to set the desired theme would be helpful.
Using the "current" theme won't help, since every user in a team might use a different theme.

@Sharparam

This comment has been minimized.

Copy link

@Sharparam Sharparam commented Oct 18, 2019

By current theme I mean the report automatically adjusting to the theme used when viewing it. But perhaps this is a limitation in iframes?

@danielpalme

This comment has been minimized.

Copy link
Contributor

@danielpalme danielpalme commented Oct 18, 2019

Correct. That's not possible with iframes.

@phatcher

This comment has been minimized.

Copy link

@phatcher phatcher commented Nov 1, 2019

Might be more of a general yaml question, but how can I set this variable in a template - I have a common pipeline for my library projects and it doesn't like having 'variables' in the template file.

@mnoreke

This comment has been minimized.

Copy link

@mnoreke mnoreke commented Nov 1, 2019

Might be more of a general yaml question, but how can I set this variable in a template - I have a common pipeline for my library projects and it doesn't like having 'variables' in the template file.

I looked at my code to see how I solved it, and I set the variable in each caller. A quick search found this article though that might help. Look at the section titled "Variable re-use" and see if it might work for you. :)

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.