FCC shows zero coverage when test fails during [Setup] - due to dll copying

[SigurdJanson]

Installed product versions

• Visual Studio: Microsoft Visual Studio Community 2019, Version 16.10.3
• This extension: 1.1.141
• Nunit 3.17.0

Description

FCC shows 0% code coverage. The test fixture is completely ignored.

Steps to recreate

The cause seems to be a line of code in the [Setup] method.

1. Add a setup method to the test class
2. Add throw new Exception().
3. Run test
4. FCC removes all coverage and shows 0%.

Current behavior

1. In the setup method I copied a file to the temporary folder to run some tests on this file.
2. Once the situation occurred that the file was already in the temp folder and the test failed because of the exception during set up (because the file could not be copied since it was already there).
3. Then FCC removed all coverage.
4. When I comment that line of code out, FCC shows 53.7% coverage again.
5. But that seems the only thing that works. This did not:
   • Comment that line out, run a test, and remove the comment: did not work.
   • Change the statement: e.g. I replace the constant path const string BasePath with the string "...\...\folderlocation": did not work.

I finally tracked the error down in my case. FCC seems to run the tests itself and causes an exception every time it runs the test because AppDomain.CurrentDomain.BaseDirectory seems to be different from the path of the testing environment.

[SetUp]
public void SetUp()
{
       string BasePath = Path.GetFullPath(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "..\\..\\TestDataFiles"));
       File.Copy(Path.Combine(BasePath, "RecipeFile.json"), Path.Combine(TestingPath, "RecipeFile.json"));
       // ...

FCC tries to copy the file from "[...]\bin\Debug\TestDataFiles" which does not work in my case. Consequence: every time the setup is called it ends in an exception. Hence, none of the tests succeeds and I get no coverage at all.

Expected behavior

Shouldn't I get the code coverage regardless of the fact that tests fail or succeed? And regardless of the place where the test fails, whether in the test method or in setup/teardown?

Notes

• The option RunWhenTestsFail is set to true.

[tonyhallett]

FCC tries to copy the file from "[...]\bin\Debug\TestDataFiles" which does not work in my case. Consequence: every time the setup is called it ends in an exception. Hence, none of the tests succeeds and I get no coverage at all.

Yes that is the case and should be mentioned in the readme. I have now updated.

As per

Shouldn't I get the code coverage regardless of the fact that tests fail or succeed? And regardless of the place where the test fails, whether in the test method or in setup/teardown?

FCC will just report what the underlying coverage tool provides.
For a .NET Core project with Coverlet - 0% will be reported.
For a .NET Framework project with OpenCover - the FCC Output Window Pane reports

and the ( converted ) report FCC uses

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
<coverage line-rate="1" branch-rate="1" lines-covered="0" lines-valid="0" branches-covered="0" branches-valid="0" complexity="NaN" version="0" timestamp="1636899656">
  <sources />
  <packages />
</coverage>

[tonyhallett]

The latest version has an option AdjacentBuildOutput that should keep relative paths in tests working.