Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenCover suddenly failing to output coverage in Azure DevOps #915

Closed
Guitrum opened this issue Aug 23, 2019 · 4 comments

Comments

@Guitrum
Copy link

@Guitrum Guitrum commented Aug 23, 2019

My Framework

  • .NET 2
  • .NET 3.5
  • .NET 4
  • .NET 4.5
  • .NET 4.6
  • .NET 4.6.1
  • .NET 4.6.2
  • .NET 4.7
  • .NET 4.7.1
  • .NET 4.7.2
  • .NET Core 1.0.0
  • .NET Core 2.0.0
  • Something else

My Environment

  • Windows 7 or below (not truly supported due to EOL)
  • Windows 8
  • Windows 8.1
  • Windows 10
  • Windows 10 IoT Core
  • Windows Server 2012
  • Windows Server 2012 R2
  • Windows Server 2016

I have already...

  • repeated the problem using the latest stable release of OpenCover.
  • reviewed the usage guide and usage document.
  • have looked at the opencover output xml file in an attempt to resolve the issue.
  • reviewed the current issues to check that the issue isn't already known.

My issue is related to (check only those which apply):

  • no coverage being recorded
  • 32 or 64 bit support
  • feature request

Expected Behavior

I have been using OpenCover for about a year now with no issues. I am running it via the following powershell script:

$info = New-Object System.Diagnostics.ProcessStartInfo
$info.FileName = "C:\Users\devUser\.nuget\packages\opencover\4.7.922\tools\OpenCover.Console.exe"
$info.RedirectStandardError = $true
$info.RedirectStandardOutput = $true
$info.UseShellExecute = $false
$info.Arguments = "`
                -target:""C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"" `
                -targetargs:""C:\agent\_work\3\s\UnitTests\bin\Debug\UnitTests.dll"" `
                -output:""C:\agent\_work\3\s\opencover.xml"" `
                -filter:""+[*]*.Service.* +[*]*.Model.* +[*]*.Repository.*"" `
                -excludebyfile:*""*\*Designer.cs"" `
                -mergebyhash `
                -skipautoprops `
                -returntargetcode `
                -register:user"

$exec = New-Object System.Diagnostics.Process
$exec.StartInfo = $info
$exec.Start() | Out-Null
$exec.StandardOutput.ReadToEnd() | Out-File -FilePath "C:\agent\_work\3\s\log.txt"
$exec.StandardError.ReadToEnd() | Out-File -FilePath "C:\agent\_work\3\s\log_errors.txt"

When running it locally against my code, I get the expected output:

Executing: C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
Microsoft (R) Test Execution Command Line Tool Version 16.2.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
  û GetBookByBookID [636ms]
  û SelectBooksByUser [132ms]
  û SelectAllBooks [64ms]
  û GetBookIDByCoinID [70ms]
  û SelectCoinsByBook [79ms]
  û PingS3 [922ms]
  û GetCoinByCoinID [140ms]
  û SelectCoinsBySearch [66ms]
  û AddAndRemoveCoin [383ms]
  û SelectAllCoins [170ms]
  û SelectAllCoinsByBookByUser [135ms]
  û SelectOwnedCoinsByBook [128ms]
  û SelectAllMintmarks [63ms]
  û SelectAllYears [75ms]
  û SelectAllDenominations [63ms]
  û SelectAllLinkUserCoin [189ms]
  û SelectUsers [125ms]
  û GetUserNameByUserID [62ms]
  û GetUserIDByUserName [69ms]
  û SelectUsersByOwned [189ms]
  û SelectAllLinkUserBook [67ms]

Test Run Successful.
Total tests: 21
     Passed: 21
 Total time: 6.0269 Seconds
Committing...
Visited Classes 11 of 11 (100)
Visited Methods 57 of 57 (100)
Visited Points 336 of 336 (100)
Visited Branches 105 of 113 (92.92)

==== Alternative Results (includes all methods including those without corresponding source) ====
Alternative Visited Classes 11 of 11 (100)
Alternative Visited Methods 62 of 62 (100)

Actual Behavior

However, when running the exact same script in Azure DevOps (which is using the same laptop/user as the build agent) I get the error where no coverage results are available:

Executing: C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
Microsoft (R) Test Execution Command Line Tool Version 16.2.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
  √ GetBookByBookID [471ms]
  √ SelectBooksByUser [136ms]
  √ SelectAllBooks [62ms]
  √ GetBookIDByCoinID [62ms]
  √ SelectCoinsByBook [66ms]
  √ PingS3 [417ms]
  √ GetCoinByCoinID [133ms]
  √ SelectCoinsBySearch [66ms]
  √ AddAndRemoveCoin [413ms]
  √ SelectAllCoins [134ms]
  √ SelectAllCoinsByBookByUser [142ms]
  √ SelectOwnedCoinsByBook [129ms]
  √ SelectAllMintmarks [63ms]
  √ SelectAllYears [64ms]
  √ SelectAllDenominations [61ms]
  √ SelectAllLinkUserCoin [148ms]
  √ SelectUsers [134ms]
  √ GetUserNameByUserID [63ms]
  √ GetUserIDByUserName [61ms]
  √ SelectUsersByOwned [225ms]
  √ SelectAllLinkUserBook [64ms]

Test Run Successful.
Total tests: 21
     Passed: 21
 Total time: 4.8670 Seconds
Committing...
No results, this could be for a number of reasons. The most common reasons are:
    1) missing PDBs for the assemblies that match the filter please review the
    output file and refer to the Usage guide (Usage.rtf) about filters.
    2) the profiler may not be registered correctly, please refer to the Usage
    guide and the -register switch.

Steps to reproduce the problem:

  1. Update the above powershell script to have valid paths for your project/machine
  2. Run the script locally and look at the log.txt to see the expected output
  3. Now run that script in an Azure DevOps build and check the log.txt (the script will not fail and will have an exit code of 0, but the opencover.xml file will have no results)
  • reviewed the usage guide and usage document again because we know you probably didn't do it the first time.
  • sample code or link to repository/gist is available (support is much faster this way)
@SteveGilham

This comment has been minimized.

Copy link
Contributor

@SteveGilham SteveGilham commented Aug 29, 2019

There's a similar Azure DevOps issue mentioned in this comment; the work-round in that case was to use -register:Path32 or -register:Path64 as appropriate.

As a guess, I wonder whether the Azure process is running with sufficient privilege to register a COM object, even as user, or whether it's too heavily sandboxed for that.

@Guitrum

This comment has been minimized.

Copy link
Author

@Guitrum Guitrum commented Aug 29, 2019

There's a similar Azure DevOps issue mentioned in this comment; the work-round in that case was to use -register:Path32 or -register:Path64 as appropriate.

As a guess, I wonder whether the Azure process is running with sufficient privilege to register a COM object, even as user, or whether it's too heavily sandboxed for that.

@SteveGilham Holy cow that worked! After months of using -register:user without fail, switching to -register:Path32 was all that was needed. Thank you for your help!

@SteveGilham

This comment has been minimized.

Copy link
Contributor

@SteveGilham SteveGilham commented Aug 29, 2019

Looking at this recent cluster of reports, a recent change in behaviours may well have been the same issue as #914 and #911 -- problems with registration under .net 4.8

@SteveGilham SteveGilham referenced this issue Aug 29, 2019
5 of 5 tasks complete
@DirkFEng

This comment has been minimized.

Copy link

@DirkFEng DirkFEng commented Sep 2, 2019

Thanks for the Hint to -registerPath:Path4 - now it works again on .NET 4.8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.