Repository navigation
Assert.Inconclusive() reports failed when timeout used #3632
Description
Activity
I cannot answer for the Resharper test runner, as I don't know anything about how that is implemented. But if I run the tests (using NUnit 3.12.0 and NUnit3TestAdapter 3.17.0 and VS Version 16.8.0 Preview 2.1) then I get the following
Which versions are you using and what information is there in the failed test (error message and e.g. stacktace)
Versions
Visual Studio - 16.7.3
Nuget
- Microsoft.NET.Test.Sdk - 16.7.1
- NUnit - 3.12.1
- NUnit3TestAdapter - 3.17.0
Stack Trace
System.AggregateException : One or more errors occurred. (Rethrown) ----> NUnit.Framework.Internal.NUnitException : Rethrown ----> NUnit.Framework.InconclusiveException : No message provided at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout) at NUnit.Framework.Internal.Commands.TimeoutCommand.Execute(TestExecutionContext context) --NUnitException at NUnit.Framework.Internal.Reflect.InvokeMethod(MethodInfo method, Object fixture, Object[] args) at NUnit.Framework.Internal.MethodWrapper.Invoke(Object fixture, Object[] args) at NUnit.Framework.Internal.Commands.TestMethodCommand.InvokeTestMethod(TestExecutionContext context) at NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext context) at NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext context) at NUnit.Framework.Internal.Commands.BeforeTestCommand.Execute(TestExecutionContext context) at NUnit.Framework.Internal.Commands.TimeoutCommand.<>c__DisplayClass2_0.<Execute>b__0() at System.Threading.Tasks.Task`1.InnerInvoke() at System.Threading.Tasks.Task.<>c.<.cctor>b__274_0(Object obj) at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location where exception was thrown --- at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) --InconclusiveException at NUnit.Framework.Assert.Inconclusive(String message, Object[] args) at NUnit.Framework.Assert.Inconclusive() at MyLibrary.Tests.MyTest.Fails() in C:\projects\json-everything\JsonPath.Tests\InlineGoessnerTests.cs:line 263Looks like it might be VS, but I prefer not to be on preview versions.
Looking at the stacktrace, I can see that the
InconclusiveExceptionis nested inside anAggregateException, so this could be a duplicate of #3282. Which framework are you targeting? (because I think that this might fail on dotnetcore until 3.13 is released).EDIT: Does it work if you use the prerelease build of the NUnit framework (from https://www.myget.org/feed/nunit/package/nuget/NUnit) instead of 3.12.0 ?
@mikkelbu are you talking about using the 4.0.0-alpha.1 version? You mention 3.12.0, but I'm on 3.17.0 right now.I was looking at the test adapter. I'm using the nuget.org version.I'm running Core 3.1 in my test project.
I'm not talking about the adapter, but the version of the nunit framework you are using (so replacing the "NUnit - 3.12.1" you mentioned above). If you follow the link - https://www.myget.org/feed/nunit/package/nuget/NUnit - then there are descriptions of how to install the prerelease, so that you can test if it solves your problem.
Yeah, looks to be resolved with 3.13.0 but reproducible with 3.12.0.
I've attached a sample solution to the initial post.
These tests should have the same result
Intead, the one with the timeout is marked as failed.
Resharper test runner:
VS test runner:
I was able to find a workaround:
reproduction