Debug.Assert breaks running tests in NetCore/NetStandard projects, but works fine when debugging

[jimmylewis]

Description

When test execution hits a failing System.Diagnostics.Debug.Assert(), it kills the entire test run. This prevents developers from unit testing failure code paths or from using Debug.Assert (which is a valuable tool on those code paths for catching issues beyond unit test coverage).

Running the same tests under the debugger in the VS IDE does not fail (though it does hit a breakpoint in System.Private.CoreLib.dll).

It would be preferable to not kill the test run and either:

• Fail the test with a message about the assertion failure (i.e. limit the impact to the executing test only).
• Not fail the test (as has been the behavior in NetFx), so that we can protect our failure code paths with Debug.Assert and unit test them as well.

Steps to reproduce

1. git clone https://github.com/jimmylewis/DebugAssertFailure.git
2. cd DebugAssertFailure
3. dotnet test

Or, clone the repo and run tests in VS

Expected behavior

All tests should pass.

Actual behavior

The active test run was aborted. Reason: Assertion Failed
at DebugAssertFailure.UnitTest1.TestMethod1() in C:\temp\DebugAssertFailure\DebugAssertFailure\UnitTest1.cs:line 12
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at ...

Environment

    <PackageReference Include="MSTest.TestAdapter" Version="1.3.2" />
    <PackageReference Include="MSTest.TestFramework" Version="1.3.2" />

[karanjitsingh]

@jimmylewis, this is not related to test. It's the default behavior in dotnet itself, if you run the following console app in cli with dotnet ConsoleApp5.dll things will break and not move to the next line.

ConsoleApp5.zip

[jimmylewis]

@karanjitsingh Is there something that testfx can do to not end the test run and allow the rest of the tests a chance to run? It would be more acceptable to mark a test as failed, but in my solution this happens on around the 5th test of 200, so I can't even run most of my tests without removing the assert.

[karanjitsingh]

Problem here is that Debug.Assert can't be caught, its purpose is to throw no matter what.