TimeoutAttribute makes all Assertions count as failure

[fbrosseau]

Hello

TimeoutAttribute(via TimeoutCommand) wraps the tests in a Task.Wait call under netcore (because Thread.Abort is no longer supported API), here:

nunit/src/NUnitFramework/framework/Internal/Commands/TimeoutCommand.cs

Line 102 in d8490ec

if (!Task.Run(() => context.CurrentResult = innerCommand.Execute(context)).Wait(_timeout))

Using any of the 'flow-control' assertions like Assert.Ignore will throw their respective special-case Exception (here IgnoreException), which Task.Wait will wrap in an AggregateException, which is not handled by ValidateAndUnwrap here

nunit/src/NUnitFramework/framework/Internal/Results/TestResult.cs

Line 564 in c3caee3

if ((ex is NUnitException || ex is TargetInvocationException) && ex.InnerException != null)

This will result in the test counting as full-blown failure due to unhandled exception.

This simple test will do it:

[Test, Timeout(1234)]
public void TimeoutBug()
{
    Assert.Ignore("a");
}

I believe it should be safe to change ValidateAndUnwrap to handle AggregateExceptions the same way it does for TargetInvocationExceptions. It may also be beneficial to dive into the exception recursively instead of only once.

[airbreather]

Another option:

1. originalTask = the Task.Run we do now
2. delayTask = a Task.Delay(_timeout)
3. Task.WhenAny(originalTask, delayTask).GetAwaiter().GetResult() gives you the first Task to complete.
   • if it's delayTask, then we hit the timeout block
   • otherwise, call .GetAwaiter().GetResult() on it, which should do the right thing

[provegard]

Came here to report this exact issue. We want to use an assembly-wide Timeout attribute, but tests that use Assert.Ignore start to fail. Any workaround except not using Assert.Ignore?

[roji]

Ran into this too.