RemotingException in NUnit.Engine.Runners.ProcessRunner.UnloadPackage

[CharliePoole]

@vitaliusvs commented on Mon Jun 27 2016

NUnit version 3.4.0
mono 4.4.0-182
CentOS 6.8

I'm running a lot of tests from a single assembly and unfortunately unable to isolate small subset of tests that are causing nunit to crash after all tests are complete and nunit is about to finish. There is stack trace of the crash:

• Assertion: should not be reached at sgen-scan-object.h:101

Stacktrace:

Native stacktrace:

    /opt/xstream-mono/bin/mono() [0x49df85]
    /lib64/libpthread.so.0(+0xf7e0) [0x7fa466a187e0]
    /lib64/libc.so.6(gsignal+0x35) [0x7fa4664915e5]
    /lib64/libc.so.6(abort+0x175) [0x7fa466492dc5]
    /opt/xstream-mono/bin/mono() [0x6422f2]
    /opt/xstream-mono/bin/mono() [0x64207c]
    /opt/xstream-mono/bin/mono() [0x64223c]
    /opt/xstream-mono/bin/mono() [0x5ffa95]
    /opt/xstream-mono/bin/mono() [0x5f1c8e]
    /opt/xstream-mono/bin/mono() [0x5f4644]
    /opt/xstream-mono/bin/mono() [0x5f49d4]
    /opt/xstream-mono/bin/mono() [0x5f4b9b]
    /opt/xstream-mono/bin/mono() [0x5f4f23]
    /opt/xstream-mono/bin/mono() [0x5a7c5a]
    /opt/xstream-mono/bin/mono() [0x6398ca]
    /lib64/libpthread.so.0(+0x7aa1) [0x7fa466a10aa1]
    /lib64/libc.so.6(clone+0x6d) [0x7fa466547aad]

Debug info from gdb:

=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.

System.Runtime.Remoting.RemotingException: Tcp transport error.

Server stack trace:
at System.Runtime.Remoting.Channels.Tcp.TcpMessageIO.ReceiveMessageStatus (System.IO.Stream networkStream, System.Byte[] buffer) <0x41be25a0 + 0x000d3> in :0
at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage (IMessage msg, ITransportHeaders requestHeaders, System.IO.Stream requestStream, ITransportHeaders& responseHeaders, System.IO.Stream& responseStream) <0x41bea010 + 0x001f3> in :0
at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage (IMessage msg) <0x41be94d0 + 0x003d7> in :0

Exception rethrown at [0]:
---> System.Runtime.Remoting.RemotingException: Connection closed
at System.Runtime.Remoting.Channels.Tcp.TcpMessageIO.StreamRead (System.IO.Stream networkStream, System.Byte[] buffer, Int32 count) <0x41be28f0 + 0x0009f> in :0
at System.Runtime.Remoting.Channels.Tcp.TcpMessageIO.ReceiveMessageStatus (System.IO.Stream networkStream, System.Byte[] buffer) <0x41be25a0 + 0x00053> in :0
--- End of inner exception stack trace ---
at (wrapper managed-to-native) System.Object:__icall_wrapper_mono_remoting_wrapper (intptr,intptr)
at (wrapper remoting-invoke) NUnit.Engine.Agents.RemoteTestAgent:Unload ()
at NUnit.Engine.Runners.ProcessRunner.UnloadPackage () <0x41bdcf90 + 0x0005a> in :0

---

@CharliePoole commented on Mon Jun 27 2016

Are you using the NUnit console runner? Does it produce any output at all?

---

@vitaliusvs commented on Tue Jun 28 2016

Yes, I'm using nunit3-console.exe. Tests run correctly, but nunit process crashes when nunit is about to finish. I found the test that was causing the trouble. The test was starting quartz scheduler http://www.quartz-scheduler.org/ in test fixture setup. I fixed the test by shutting down quartz scheduler in test fixture tear down. However I still think nunit process should not crash once nunit agent fails.

---

@CharliePoole commented on Tue Jun 28 2016

This may be the same problem as #1628, assuming your scheduler is running on a thread that didn't allow the agent process to unload and terminate. At a minimum, I suspect it's related, so let's wait till 3.4.1 comes out today or tomorrow.

---

@CharliePoole commented on Mon Aug 29 2016

@vitaliusvs Have you tried this on 3.4.1? Did it solve the problem?

---

@CharliePoole commented on Mon Aug 29 2016

This should have been moved to the nunit-console repo when we split repos. Doing it now.

[CharliePoole]

Marking this as a bug, but also as 'confirm' since we have to confirm that it's our bug and not mono's.

I have run tests using various mono releases and determined that the problem first appears in mono 4.4.0.

[jnm2]

@nunit/engine-team This looks similar to some of the stack traces we were seeing that #223 fixed, but I'm not sure if it's the same problem. I will take this on when I have a chance in case it's related.

[ChrisMaddock]

@vitaliusvs This might be fixed by @BlythMeister's temporary fix in #370. (Or @jnm2's work in #223 also!)

Would you try the master build, version 3.9.0-dev-03932 or later. You can find this on out MyGet Feed.

Let us know if you have a different problem.

[ChrisMaddock]

Closing this issue based on the above. We can reopen if someone re-encounters this problem. 🙂