Assert.Inconclusive() reports failed when timeout used

[gregsdennis]

These tests should have the same result

[Test]
public void Inconclusive()
{
    Assert.Inconclusive();
}

[Test]
[Timeout(1000)]
public void Fails()
{
    Assert.Inconclusive();
}

Intead, the one with the timeout is marked as failed.

Resharper test runner:

VS test runner:

---

I was able to find a workaround:

using var cts = new CancellationTokenSource(1000);
Task.Run(() => yourCode.Check(), cts.Token).Wait(cts.Token);

Assert.Inconclusive();

reproduction

[mikkelbu]

I cannot answer for the Resharper test runner, as I don't know anything about how that is implemented. But if I run the tests (using NUnit 3.12.0 and NUnit3TestAdapter 3.17.0 and VS Version 16.8.0 Preview 2.1) then I get the following

Which versions are you using and what information is there in the failed test (error message and e.g. stacktace)

[gregsdennis]

Versions

Visual Studio - 16.7.3

Nuget

• Microsoft.NET.Test.Sdk - 16.7.1
• NUnit - 3.12.1
• NUnit3TestAdapter - 3.17.0

Stack Trace

System.AggregateException : One or more errors occurred. (Rethrown)
  ----> NUnit.Framework.Internal.NUnitException : Rethrown
  ----> NUnit.Framework.InconclusiveException : No message provided
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout)
   at NUnit.Framework.Internal.Commands.TimeoutCommand.Execute(TestExecutionContext context)
--NUnitException
   at NUnit.Framework.Internal.Reflect.InvokeMethod(MethodInfo method, Object fixture, Object[] args)
   at NUnit.Framework.Internal.MethodWrapper.Invoke(Object fixture, Object[] args)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.InvokeTestMethod(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.BeforeTestCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.TimeoutCommand.<>c__DisplayClass2_0.<Execute>b__0()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__274_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--InconclusiveException
   at NUnit.Framework.Assert.Inconclusive(String message, Object[] args)
   at NUnit.Framework.Assert.Inconclusive()
   at MyLibrary.Tests.MyTest.Fails() in C:\projects\json-everything\JsonPath.Tests\InlineGoessnerTests.cs:line 263

[gregsdennis]

Looks like it might be VS, but I prefer not to be on preview versions.

[mikkelbu]

Looking at the stacktrace, I can see that the InconclusiveException is nested inside an AggregateException, so this could be a duplicate of #3282. Which framework are you targeting? (because I think that this might fail on dotnetcore until 3.13 is released).

EDIT: Does it work if you use the prerelease build of the NUnit framework (from https://www.myget.org/feed/nunit/package/nuget/NUnit) instead of 3.12.0 ?

[gregsdennis]

@mikkelbu are you talking about using the 4.0.0-alpha.1 version? You mention 3.12.0, but I'm on 3.17.0 right now. I was looking at the test adapter. I'm using the nuget.org version.

I'm running Core 3.1 in my test project.

[mikkelbu]

I'm not talking about the adapter, but the version of the nunit framework you are using (so replacing the "NUnit - 3.12.1" you mentioned above). If you follow the link - https://www.myget.org/feed/nunit/package/nuget/NUnit - then there are descriptions of how to install the prerelease, so that you can test if it solves your problem.

[gregsdennis]

Yeah, looks to be resolved with 3.13.0 but reproducible with 3.12.0.

I've attached a sample solution to the initial post.

[mikkelbu]

I'll close this as a duplicate of #3282. This should be solved with 3.13.0 which hopefully is out soon.