SocketException when exploring a test assembly

[yaakov-h]

I'm upgrading my projects from NUnit engine 3.4 to 3.6, using framework 3.6 before and after.

When exploring an assembly for tests, I get the following exception:

>"C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" --explore C:\Temp\NUnitRepro\NUnitRepro\bin\Debug\NUnitRepro.dll
NUnit Console Runner 3.6.0
Copyright (C) 2017 Charlie Poole

Runtime Environment
   OS Version: Microsoft Windows NT 10.0.14393.0
  CLR Version: 4.0.30319.42000

Test Files
    C:\Temp\NUnitRepro\NUnitRepro\bin\Debug\NUnitRepro.dll

System.Net.Sockets.SocketException (0x80004005): An existing connection was forcibly closed by the remote host

Server stack trace:
   at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
   at System.Runtime.Remoting.Channels.SocketStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Runtime.Remoting.Channels.SocketHandler.ReadFromSocket(Byte[] buffer, Int32 offset, Int32 count)
   at System.Runtime.Remoting.Channels.SocketHandler.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.Runtime.Remoting.Channels.SocketHandler.ReadAndMatchFourBytes(Byte[] buffer)
   at System.Runtime.Remoting.Channels.Tcp.TcpSocketHandler.ReadAndMatchPreamble()
   at System.Runtime.Remoting.Channels.Tcp.TcpSocketHandler.ReadVersionAndOperation(UInt16& operation)
   at System.Runtime.Remoting.Channels.Tcp.TcpClientSocketHandler.ReadHeaders()
   at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage(IMessage msg, ITransportHeaders requestHeaders, Stream requestStream, ITransportHeaders& responseHeaders, Stream& responseStream)
   at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at NUnit.Engine.ITestAgent.Stop()
   at NUnit.Engine.Runners.ProcessRunner.Dispose(Boolean disposing)
   at NUnit.Engine.Runners.AbstractTestRunner.Dispose()
   at NUnit.Engine.Runners.MasterTestRunner.Dispose(Boolean disposing)
   at NUnit.Engine.Runners.MasterTestRunner.Dispose()
   at NUnit.ConsoleRunner.ConsoleRunner.ExploreTests(TestPackage package, TestFilter filter)
   at NUnit.ConsoleRunner.Program.Main(String[] args)

I do not get this exception when using --inprocess, nor when omitting --explore.

Here's a repro case which fails reliably for me:

using System.Collections.Generic;
using NUnit.Framework;

namespace NUnitRepro
{
	class MyTest
	{
		[TestCaseSource(nameof(TestCases))]
		public void AssertDirectoryExists(string directory)
		{
			DirectoryAssert.Exists(directory);
		}

		public static IEnumerable<TestCaseData> TestCases
		{
			get
			{
				yield return new TestCaseData(TestContext.CurrentContext.TestDirectory);
			}
		}
	}
}

[CharliePoole]

You should definitely not be getting a socket exception.

However, your repro case is equivalent to just throwing an exception! Since the TestCases static member is being called outside of running the test, it's guaranteed that the current context will be null.

This is actually a duplicate of issue #1824

[yaakov-h]

The interesting thing is that the context isn't null. If you do File.WriteAllText(@"C:\TestDir.txt", TestContext.CurrentContext.TestDirectory) from the test case source, then the test directory is written to that file before the process crashes.

[jnm2]

We believe this was fixed by nunit/nunit-console#223 which will be released as part of NUnit.Console 3.7. If you'd like to confirm sooner that the issue is fixed, please try 3.7.0-dev-03641 from the https://ci.appveyor.com/nuget/nunit-console NuGet feed.

If it turns out that this does not fix your issue, please open a new issue at https://github.com/nunit/nunit-console/issues.

[yaakov-h]

Thanks @jnm2. I'll do some testing early next week to verify.

[yaakov-h]

Seems to be all good. Thanks again @jnm2 and team.