dotnet test --collect "Code Coverage;Format=cobertura doesn't seem to create a coverage file

[veikkoeeva]

The instructions

dotnet new mstest -o mstests
cd mstests
dotnet test --collect "Code Coverage;Format=cobertura"

do not seem to work. It creates a file with <Project Sdk="MSTest.Sdk/3.5.1"> that does not work generate a coverage file on my computer. The output to console is also just

dotnet test --collect "Code Coverage;Format=cobertura"
Restore complete (0,8s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  mstests succeeded (2,0s) → bin\Debug\net9.0\mstests.dll
  mstests test succeeded (0,8s)

Test summary: total: 1; failed: 0; succeeded: 1; skipped: 0; duration: 0,3s
Build succeeded in 4,1s

I have a bit more elaborate GitHub setup at Lumoin/Verifiable#377 and the issue seems to be the same that no output file is generated (at https://github.com/Lumoin/Verifiable/actions/runs/11203930073/job/31141727980#step:14:1, the command line ecludes XUnit still even if they aren't present, but it doesn't seem to get even that far due to there not being a file to read).

The global.json is

{
    "sdk": {
        "allowPrerelease": true,
        "version": "9.0.100-rc.1.24452.12"
    },
    "msbuild-sdks": {
        "MSTest.Sdk": "3.6.1"        
    }
}

I am not sure if this is the right place to note this, but I'm thinking at least the instructions

dotnet new mstest -o mstests
cd mstests
dotnet test --collect "Code Coverage;Format=cobertura"

should work so that a coverage file will be generated or then maybe some reason given if not. My OS is Windows 11 with the latest patches, with fi-FI settings,

> dotnet --version 
> 9.0.100-rc.1.24452.12

[jakubch1]

@Evangelink could you please help here?

[Evangelink]

Hi @veikkoeeva,

Thanks for reporting the issue. On .NET 9, we are making MSTest runner enabled by default (and we bind dotnet test experience to it) which means that some "extensions" are no longer working the same way they used to.

Coverlet is one of these extensions. As described here https://learn.microsoft.com/dotnet/core/testing/unit-testing-platform-extensions-code-coverage#coverlet, you will need to use coverlet global tool instead.

If you are not interested in benefiting from the new runner, you can revert to VSTest runner by adding (<UseVSTest>true</UseVSTest>) as documented on our SDK page (https://learn.microsoft.com/dotnet/core/testing/unit-testing-mstest-sdk#select-the-runner). Alternatively, you could keep the new runner but disconnect it from the dotnet test experience (so keep using VSTest when you do dotnet test by adding <TestingPlatformDotnetTestSupport>false</TestingPlatformDotnetTestSupport> to your project).

[veikkoeeva]

Hi, thanks!

Some may be here to wonder about .coverage file. That neither is produced and that was a partial fudging factor for me.

In any event, I tried with dotnet tool run dotnet-coverage collect --output coverage/report.cobertura.xml --output-format cobertura "dotnet test --no-build" for instance and this produced a coverage report Cobertura format as far as I can see with a naked eye.

I did not use the .csproj options, so I suppose I'm opting in to the latest and greates of bleeding edge as of this writing.

<edit: So, I may have not actually understood how to do this correctly. @Evangelink would it be possible to have an example of a working command line on how to get a real cobertura file from which to generate the report or a confirmation this the way? I'd be OK with other formats too to get a coverage file if that's possible.

<edit 2: Sorry for the trouble. I think this is the way indeed. I just hit some (many) red lines on e.g. compiled regexes and stuff. :)

<edit 3: For others, a more direct way, indeed without involving "dotnet test" would be dotnet tool run dotnet-coverage collect --output coverage/report.cobertura.xml --output-format cobertura test/Project.Tests/bin/Debug/net9.0/Projects.Tests.exe

[jakubch1]

Hi @veikkoeeva,
For new MSTest runner you can use dotnet-coverage tool as you suggested. You can also run application and provide:

test/Project.Tests/bin/Debug/net9.0/Projects.Tests.exe --coverage --coverage-output report.cobertura.xml --coverage-output-format cobertura

Please check scenarios here: https://github.com/microsoft/codecoverage/tree/main/samples/Algorithms
Especially Scenario 04 and Scenario 07