Failing test never completes

[mbp]

I have a strange issue after updating to "xUnit.net Test Support 2.1.6"

I use Visual Studio, ReSharper 8.2.3, xUnit 1.9.2 and FluentAssertions 3.3.0.

The following test never completes:

[Fact]
public void Failing_Test_Never_Completes()
{
    var oneList = new List<MyDto>
    {
        new MyDto
        {
            Id = 42,
        }
    };
    var secondList = new List<MyDto>
    {
        new MyDto
        {
            Id = 44,
        },
    };

    oneList.ShouldAllBeEquivalentTo(secondList);
}

When I debug the test, it appears a TargetInvocationException is thrown from xUnit, which I tracked down to IndexOutOfRangeException in the following stack trace:

at XunitContrib.Runner.ReSharper.RemoteRunner.AbstractionExtensions.ConvertExceptions(IFailureInformation failure, String& simplifiedMessage)
at XunitContrib.Runner.ReSharper.RemoteRunner.TestExecutionVisitor.Visit(ITestFailed testFailed)
at Xunit.TestMessageVisitor.DoVisit[TMessage](IMessageSinkMessage message, Func`2 callback)
at Xunit.TestMessageVisitor.OnMessage(IMessageSinkMessage message)
at Xunit.TestMessageVisitor`1.OnMessage(IMessageSinkMessage message)
at XunitContrib.Runner.ReSharper.RemoteRunner.TestExecutionVisitor.OnMessage(IMessageSinkMessage message)
at Xunit.TestClassCallbackHandler.OnTest(XmlNode xml)
at Xunit.TestClassCallbackHandler.OnXmlNode(XmlNode xml)
at Xunit.XmlNodeCallbackHandler.System.Web.UI.ICallbackEventHandler.RaiseCallbackEvent(String eventArgument)
at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

In the call stack, I found a variable with the following XML, thought it might be relevant.

<test name="ConsoleApplication7.MyTest.Failing_Test_Never_Completes" type="ConsoleApplication7.MyTest" method="Failing_Test_Never_Completes" result="Fail" time="0.698"><failure exception-type="Xunit.Sdk.AssertException"><message>Expected item[0].Id to be 44, but found 42.

With configuration:
- Use declared types and members
- Compare enums by value
- Match member by name (or throw)
- Be strict about the order of items in byte arrays
</message><stack-trace>   ved FluentAssertions.Execution.LateBoundTestFramework.Throw(String message) i z:\Builds\work\b7ceef20fdfcf0ee\Shared\Execution\LateBoundTestFramework.cs:linje 30
   ved FluentAssertions.Execution.TestFrameworkProvider.Throw(String message) i z:\Builds\work\b7ceef20fdfcf0ee\FluentAssertions.Net40\Execution\TestFrameworkProvider.cs:linje 42
   ved FluentAssertions.Execution.CollectingAssertionStrategy.ThrowIfAny(IDictionary`2 context) i z:\Builds\work\b7ceef20fdfcf0ee\FluentAssertions.Core\Execution\CollectingAssertionStrategy.cs:linje 50
   ved FluentAssertions.Execution.AssertionScope.Dispose() i z:\Builds\work\b7ceef20fdfcf0ee\FluentAssertions.Core\Execution\AssertionScope.cs:linje 252
   ved FluentAssertions.Equivalency.EquivalencyValidator.AssertEquality(EquivalencyValidationContext context) i z:\Builds\work\b7ceef20fdfcf0ee\FluentAssertions.Core\Equivalency\EquivalencyValidator.cs:linje 36
   ved FluentAssertions.AssertionExtensions.ShouldBeEquivalentTo[T](T subject, Object expectation, Func`2 config, String because, Object[] reasonArgs) i z:\Builds\work\b7ceef20fdfcf0ee\Shared\AssertionExtensions.cs:linje 542
   ved FluentAssertions.AssertionExtensions.ShouldBeEquivalentTo[T](T subject, Object expectation, String because, Object[] reasonArgs) i z:\Builds\work\b7ceef20fdfcf0ee\Shared\AssertionExtensions.cs:linje 503
   ved FluentAssertions.AssertionExtensions.ShouldAllBeEquivalentTo[T](IEnumerable`1 subject, IEnumerable expectation, String because, Object[] reasonArgs) i z:\Builds\work\b7ceef20fdfcf0ee\Shared\AssertionExtensions.cs:linje 567
   ved ConsoleApplication7.MyTest.Failing_Test_Never_Completes() i c:\Users\mbp\Documents\Visual Studio 2013\Projects\ConsoleApplication7\ConsoleApplication7\Program.cs:linje 34</stack-trace></failure></test>

[bradwilson]

That's probably this: xunit/xunit#375

[mbp]

I hope it doesn't mean I need to upgrade to xUnit 2, because we use SpecFlow which does not support xUnit 2 yet.

[mbp]

By the way - the test output goes perfectly fine to the ReSharper output window. It just doesn't finish.

So it would great if it could be fixed in XunitContrib.Runner.ReSharper.RemoteRunner.AbstractionExtensions.ConvertExceptions, so the exception doesn't occur.

[mbp]

Downgrading to xUnit.net Test Support 1.6.2 works.

[bradwilson]

No, it does not mean you need to upgrade to xUnit.net v2.

What it means is that the Resharper runner will want to upgrade to the v2.1 runner utility, though, to prevent this problem in the future.

[citizenmatt]

I'll take a look to see if I can better handle exceptions, too. I'd rather it fail fast than sit there without completion. Is 2.1 baked yet?

[bradwilson]

No, I'm expecting 2.1 to have a long tail, since right now its primary feature is DNX (ASP.NET v5) support. If you want, I can back-port this to create a 2.0.1.

[chrisjansson]

I'm using ReSharper 9.1 with xUnit Test Support 2.1.6 and this is currently causing me quite a bit of grief. Is there anything I can to to help or at least work around the issue?

[mbp]

@chrisjansson I downgraded to 1.6.2 as a work around.

Although it is not simple to downgrade ReSharper extensions, I had to manually change some stuff in AppData.

[chrisjansson]

@mbp The earliest version with ReSharper 9 support is 2.0.0 unfortunately.

[citizenmatt]

I've just pushed 2.1.7 to the extension gallery, with a fix for this issue, so you shouldn't need to install an old version now.

[chrisjansson]

Thanks, works great!