Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish code coverage task overrides ReportGenerator filters #10354

Closed
nilfdev opened this issue May 10, 2019 · 16 comments
Closed

Publish code coverage task overrides ReportGenerator filters #10354

nilfdev opened this issue May 10, 2019 · 16 comments

Comments

@nilfdev
Copy link

@nilfdev nilfdev commented May 10, 2019

Required Information

Type: bug
Enter Task Name: PublishCodeCoverageResults@1

Environment

  • Server - Azure Pipelines
  • Agent - Private
  • dotnet-reportgenerator-cli Version="4.1.5"

Issue description

I am configuring azure CI pipeline. And I've added following steps:

  • test
  • report generator
  • publish code coverage

ReportGenerator is the well know tool https://github.com/danielpalme/ReportGenerator
that exists in the Azure Devops marketplace
image

in the ReportGenerator I apply filters to exclude test projects from the total test coverage report and exclude extra xUnit assemblies from analysis.

image

In the console log I see that following filters were applied:

-assemblyfilters:-[.tests] -[.Tests] -[xunit.]* -[.Test] -[Xunit.]*
-classfilters:-[.tests] -[.Tests] -[xunit.]* -[.Test] -[Xunit.]*
-filefilters:-[.tests] -[.Tests] -[xunit.]* -[.Test] -[Xunit.]*

as you also can see, on the next step Publishing code coverage task starts and it re-generates summary report one more time without applying the assembly filtering.
it became like this after integration with reportGenerator tool
https://docs.microsoft.com/en-us/azure/devops/release-notes/2019/sprint-150-update#cobertura-code-coverage-report-updates

how to apply filtering and exclude necessary files from the report on CI, while publishing code coverage step override the filter attributes. Can you also apply settings for filtering classes and assemblies

finale report has data that I don't really need
image

@ghost ghost added route triage labels May 10, 2019
@danielpalme
Copy link
Contributor

@danielpalme danielpalme commented May 12, 2019

Disclaimer: I'm the maintainer/author of ReportGenerator

I think the new behavior is not what users are expecting.
If you specify a "Report Directory" the users would expect to see exactly the specified report files to appear in the Code Coverage tab of the build results.
Only if the "Report Directory" is not supplied, the Publish Code Coverage Results task should generate a report by itself to enhance the user experience.

@nilfdev
To disable the regeneration of the report, you can use the following environment variable in your build:

disable.coverage.autogenerate: 'true'

@nilfdev
Copy link
Author

@nilfdev nilfdev commented May 14, 2019

@danielpalme yes.
disable.coverage.autogenerate fixes the problem.
it is not obvious setting.

@PBoraMSFT
Copy link
Member

@PBoraMSFT PBoraMSFT commented May 21, 2019

@nilfdev - thank you for reporting this issue. Adding first-class support for exclusion filters is on the backlog. The suggestion that @danielpalme has made can be used as a workaround in the meantime.

@mnoreke
Copy link

@mnoreke mnoreke commented Jul 2, 2019

As I have struggled with this being new to Pipelines, I want to expand on @danielpalme 's suggestion. I tried passing it every which way as a parameter, but it needed to be set at the start of my Yaml file in the following block.

I have it just before the steps, and am listing that for someone else who might need a nudge in the right direction.

variables:
disable.coverage.autogenerate: 'true'

steps:

@Sharparam
Copy link

@Sharparam Sharparam commented Oct 17, 2019

The auto-generated reports don't seem to respect the current theme set in the DevOps GUI. Is this intended? There doesn't seem to be a way to configure the theme used by the auto-generated report, so is the only way to disable autogeneration using above methods, and keep generating it manually?

@danielpalme
Copy link
Contributor

@danielpalme danielpalme commented Oct 17, 2019

@Sharparam
An option to set the desired theme would be helpful.
Using the "current" theme won't help, since every user in a team might use a different theme.

@Sharparam
Copy link

@Sharparam Sharparam commented Oct 18, 2019

By current theme I mean the report automatically adjusting to the theme used when viewing it. But perhaps this is a limitation in iframes?

@danielpalme
Copy link
Contributor

@danielpalme danielpalme commented Oct 18, 2019

Correct. That's not possible with iframes.

@phatcher
Copy link

@phatcher phatcher commented Nov 1, 2019

Might be more of a general yaml question, but how can I set this variable in a template - I have a common pipeline for my library projects and it doesn't like having 'variables' in the template file.

@mnoreke
Copy link

@mnoreke mnoreke commented Nov 1, 2019

Might be more of a general yaml question, but how can I set this variable in a template - I have a common pipeline for my library projects and it doesn't like having 'variables' in the template file.

I looked at my code to see how I solved it, and I set the variable in each caller. A quick search found this article though that might help. Look at the section titled "Variable re-use" and see if it might work for you. :)

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops

@github-actions
Copy link

@github-actions github-actions bot commented Oct 11, 2020

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

@github-actions github-actions bot added the stale label Oct 11, 2020
@ardalis
Copy link

@ardalis ardalis commented Oct 13, 2020

This is still an issue. Running into it now with a client.

@github-actions github-actions bot removed the stale label Oct 13, 2020
@nishantms
Copy link
Contributor

@nishantms nishantms commented Oct 19, 2020

@ardalis
Can you check whether the workaround suggested by the @danielpalme works for you.

you can use the following environment variable in your build:

disable.coverage.autogenerate: 'true'

This issue is still in our backlogs.

@ardalis
Copy link

@ardalis ardalis commented Oct 19, 2020

I can confirm that that did let us have consistency between the HTML report and the generated cobertura.xml file. Thanks.

@nishantms
Copy link
Contributor

@nishantms nishantms commented Oct 28, 2020

we have put this issue in our backlogs. Will work on this as per the priority

@bl718w
Copy link

@bl718w bl718w commented Mar 15, 2021

If you want to set an environment variable inside a template, you can use the following configuration:

  • job: SetRuntimeVariables
    displayName: 'Set Runtime Variables'
    steps:
    • task: PowerShell@2
      displayName: 'Set Runtime Variables'
      inputs:
      targetType: 'inline'
      script: |
      echo '##vso[task.setvariable variable=disable.coverage.autogenerate]true'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests