Junit result xml files are not being generated for versions > 3.0.98

[kabilanmani93]

Dotnet Project

Docker base image used : mcr.microsoft.com/dotnet/sdk:6.0.405-alpine3.17

Command used:
dotnet test --test-adapter-path:. --collect "Code Coverage" /p:CollectCoverage=true /p:CoverletOutputFormat="opencover" /p:CoverletOutput=tests/TestResults /p:UseSourceLink=true --logger:"junit;LogFileName=junit-result.xml;MethodFormat=Class;FailureBodyFormat=Verbose" --logger "trx;LogFileName=testresults.trx;Parser=Legacy" --results-directory valid/

Issue:
Junit result xml files are not being generated for versions > 3.0.98

But if either we REMOVE the flag --collect "Code Coverage" then the junit xml's are generated
or
instead pass --collect "XPlat Code Coverage" then the junit xml's are generated

having --collect " code coverage" the junt result xmls are not generated

[Siphonophora]

@kabilanmani93 So, files were generated on v3.0.87?

--collect "Code Coverage" doesn't alter the xml report that we generate, right? Maybe it adds standard output that gets logged in our report?

It isn't obvious to me how these could be interfering with each other, so I think we may need an example to move this forward. Unless

[codito]

@kabilanmani93 I had a chance to debug a similar issue recently. It was with coverlet collector. The problem is Coverlet also instruments the Spekt.* dlls which changes some of the event handlers.

Is it possible to try excluding Spekt.* and JUnitXml* dlls from code coverage instrumentation?

If this doesn't work, could you share the log.* files generated with dotnet test --diag:log.txt command?

[chrisandchris]

I can confirm this behaviour:

• 🚫 with XPlat Code Coverage (coverlet collector 6.0.0) and Junit Logger 3.0.110, no xml files are generated
• 🆗 with XPlat Code Coverage (coverlet collector 6.0.0) and Junit Logger 3.0.98, the xml files are generated
• 🚫 with MS Code Coverage 17.7.2 and Junit Logger 3.0.110, no xml files are generated
• 🆗 with MS Code Coverage 17.7.2 and Junit Logger 3.0.98, the xml files are generated

We currently just made sure that 3.0.98 is installed and then we have the xml log files.

[codito]

Thanks @chrisandchris for confirmation. Is it possible to share the log.* files generated with dotnet test --diag:log.txt command?

[robrich]

Could it be related to this?: https://github.com/spekt/xunit.testlogger/issues/51#issuecomment-1872528782

[chrisandchris]

Thanks @chrisandchris for confirmation. Is it possible to share the log.* files generated with dotnet test --diag:log.txt command?

Sorry for the very late response. We stayed on 3.0.98 and then somehow we forgot to generate the logs.

Following command:

dotnet test --no-restore \
    --no-build \
    --test-adapter-path:. \
    --diag:log.txt \
    --collect:"Code Coverage;Format=cobertura" \
    --logger "junit;LogFilePath=./junit-{assembly}.xml;MethodFormat=Class;FailureBodyFormat=Verbose"

With JunitXml.TestLogger 3.0.134, i have the following log.txt: log.txt

However, I could not reliably reproduce the issue with the missing junit-xml. It worked on my machine, but our Gitlab Runner did not produce it. I'll investigate further (hopefully a bit faster...).

[timo-reymann]

@chrisandchris I find myself in the same situation, just with the Xunit XML logger. Also works locally, but not on the GitLab runners.

For some reason after a few builds it magically starts to work. In CI obviously the build only runs once, and all files are thrown aways as it runs in a container.

Maybe you have the same situation?

I can reproduce it by dotnet clean, run a build and than run dotnet test --no-build locally.

[codito]

We found that code coverage instrumentation is messing up the test result reporting callbacks. Please check if this workaround helps: #72 (comment). If anyone tries this, please report back if the workaround helped.

Tracking a fix in above issue.

[codito]

Fix released with https://www.nuget.org/packages/JunitXml.TestLogger/3.1.12.