Running under mono

[oznetmaster]

trying to run the .NET 4.5 build under mono, executing nunitlite.tests.exe results in:

mono nunitlite.tests.exe
NUnitLite 3.13.0 (.NET Framework 4.5 Debug)
Copyright (c) 2019 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: Unix 4.4.0.148
  CLR Version: 4.0.30319.17020

Test Files
    /media/sf_D_DRIVE/Projects/github/nunit/bin/debug/net45/nunitlite.tests.exe

Test Discovery
  Start time: 2019-05-22 11:19:56Z
    End time: 2019-05-22 11:19:56Z
    Duration: 0.023 seconds

System.TypeLoadException: Could not load type 'NUnit.Framework.Internal.TestSuiteResult' from assembly 'nunit.framework, Version=3.13.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb'.
  at NUnit.Framework.Internal.Execution.WorkItemBuilder.CreateWorkItem (ITest test, ITestFilter filter, Boolean recursive) <0x41997a00 + 0x00143> in <filename unknown>:0
  at NUnit.Framework.Api.NUnitTestAssemblyRunner.RunAsync (ITestListener listener, ITestFilter filter) <0x41994900 + 0x0010f> in <filename unknown>:0
  at NUnit.Framework.Api.NUnitTestAssemblyRunner.Run (ITestListener listener, ITestFilter filter) <0x41994880 + 0x0001b> in <filename unknown>:0
  at NUnitLite.TextRunner.RunTests (NUnit.Framework.Internal.TestFilter filter, IDictionary`2 runSettings) <0x41994610 + 0x00059> in <filename unknown>:0
  at NUnitLite.TextRunner.Execute () <0x41984560 + 0x0071f> in <filename unknown>:0

I have no idea what to do to debug this. This test exe runs correctly under .NET 4.5 on windows.

[oznetmaster]

Never mind.
Mono runtime was apparently still at 4.0 on machine. Updated to latest, and all is ok.

[oznetmaster]

Well, I spoke to soon. Tried running full framework tests, and got:

 mono nunitlite-runner.exe nunit.framework.tests.dll
NUnitLite 3.13.0 (.NET Framework 4.5 Debug)
Copyright (c) 2019 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: Unix 4.4.0.148
  CLR Version: 4.0.30319.42000

Test Files
    nunit.framework.tests.dll

Test Discovery
  Start time: 2019-05-22 11:47:28Z
    End time: 2019-05-22 11:47:28Z
    Duration: 0.032 seconds

Errors, Failures and Warnings

1) Invalid : /media/sf_D_DRIVE/Projects/github/nunit/bin/debug/net45/nunit.framework.tests.dll
Exception of type 'System.Reflection.ReflectionTypeLoadException' was thrown.
Could not load file or assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
Could not load file or assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
  ----> Could not load file or assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.
  ----> Could not load file or assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

Run Settings
    Number of Test Workers: 4
    Work Directory: /media/sf_D_DRIVE/Projects/github/nunit/bin/debug/net45
    Internal Trace: Off

Test Run Summary
  Overall result: Failed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2019-05-22 11:47:28Z
    End time: 2019-05-22 11:47:28Z
    Duration: 0.009 seconds

Results (nunit3) saved as /media/sf_D_DRIVE/Projects/github/nunit/bin/debug/net45/TestResult.xml

Since I know that the full test suite is run under mono for integration testing, I must be doing something wrong.

BTW, this is running under Ubuntu 16.0.4.

And:

mono -V
Mono JIT compiler version 5.20.1.19 (tarball Thu Apr 11 09:09:21 UTC 2019)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        Interpreter:   yes
        LLVM:          yes(600)
        Suspend:       hybrid
        GC:            sgen (concurrent by default)

[jnm2]

CI is using Mono 5.10 on Travis and Azure Pipelines is up to 5.11 I think.

I know that if we upgrade to Mono 5.13, a whole bunch of tests start failing with double precision issues and stack trace mismatches. (I'll have to dig up the issue I posted them in.) I saw this in CI and also on Ubuntu 18.0.4 with Mono 5.20.

Mono quirks have eaten up so much of my time that I'm being careful to remain good-humored about it. :')

Missing System.Web though makes me think you might have installed mono-devel instead of mono-complete, on the Mono install webpage's advice that it solves most missing assembly issues.

[jnm2]

Here's the 24 tests I'd expect to see fail unless you roll back to Mono 5.11:

#3230

I should reopen that issue and get to work, I guess.

[oznetmaster]

These are the ones that failed for me under 5.20 (26 of them):

1) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultiple_InvalidAssertThrowsException("AssertPassInBlock","Assert.Pass")
  StackTrace
  Expected: not null and String containing "AssertPassInBlock"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

2) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultiple_InvalidAssertThrowsException("AssertIgnoreInBlock","Assert.Ignore")
  StackTrace
  Expected: not null and String containing "AssertIgnoreInBlock"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

3) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultiple_InvalidAssertThrowsException("AssertInconclusiveInBlock","Assert.Inconclusive")
  StackTrace
  Expected: not null and String containing "AssertInconclusiveInBlock"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

4) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultiple_InvalidAssertThrowsException("AssumptionInBlock","Assume.That")
  StackTrace
  Expected: not null and String containing "AssumptionInBlock"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

5) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleErrorTests("ExceptionThrown",0)
  StackTrace
  Expected: not null and String containing "ExceptionThrown"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

6) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleErrorTests("ExceptionThrownAfterTwoFailures",2,"Failure 1","Failure 2","Simulated Error")
Multiple failures or warnings in test:
  1)   Failure 1
  Expected: 5
  But was:  4

  2)   Failure 2
  Expected: True
  But was:  False

  3) System.Exception : Simulated Error
  4)   StackTrace
  Expected: not null and String containing "ExceptionThrownAfterTwoFailures"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

7) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleErrorTests("ExceptionThrownAfterWarning",0,"WARNING","Simulated Error")
Multiple failures or warnings in test:
  1) WARNING
  2) System.Exception : Simulated Error
  3)   StackTrace
  Expected: not null and String containing "ExceptionThrownAfterWarning"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

8) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("TwoAsserts_FirstAssertFails",2,"RealPart")
Multiple failures or warnings in test:
  1)   RealPart
  Expected: 5.0d
  But was:  5.2000000000000002d

  2)   StackTrace
  Expected: not null and String containing "TwoAsserts_FirstAssertFails"
  But was:  <string.Empty>

9) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("TwoAsserts_SecondAssertFails",2,"ImaginaryPart")
Multiple failures or warnings in test:
  1)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  2)   StackTrace
  Expected: not null and String containing "TwoAsserts_SecondAssertFails"
  But was:  <string.Empty>

10) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("TwoAsserts_BothAssertsFail",2,"RealPart","ImaginaryPart")
Multiple failures or warnings in test:
  1)   RealPart
  Expected: 5.0d
  But was:  5.2000000000000002d

  2)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  3)   StackTrace
  Expected: not null and String containing "TwoAsserts_BothAssertsFail"
  But was:  <string.Empty>

11) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("NestedBlock_FirstAssertFails",3,"Expected: 5")
Multiple failures or warnings in test:
  1)   Expected: 5
  But was:  4

  2)   StackTrace
  Expected: not null and String containing "NestedBlock_FirstAssertFails"
  But was:  <string.Empty>

12) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("NestedBlock_TwoAssertsFail",3,"Expected: 5","ImaginaryPart")
Multiple failures or warnings in test:
  1)   Expected: 5
  But was:  4

  2)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  3)   StackTrace
  Expected: not null and String containing "NestedBlock_TwoAssertsFail"
  But was:  <string.Empty>

13) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("TwoNestedBlocks_FirstAssertFails",3,"Expected: 5")
Multiple failures or warnings in test:
  1)   Expected: 5
  But was:  4

  2)   StackTrace
  Expected: not null and String containing "TwoNestedBlocks_FirstAssertFails"
  But was:  <string.Empty>

14) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("TwoNestedBlocks_TwoAssertsFail",3,"Expected: 5","ImaginaryPart")
Multiple failures or warnings in test:
  1)   Expected: 5
  But was:  4

  2)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  3)   StackTrace
  Expected: not null and String containing "TwoNestedBlocks_TwoAssertsFail"
  But was:  <string.Empty>

15) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("MethodCallsFail",0,"Message from Assert.Fail")
Multiple failures or warnings in test:
  1) Message from Assert.Fail
  2)   StackTrace
  Expected: not null and String containing "MethodCallsFail"
  But was:  <string.Empty>

16) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("MethodCallsFailAfterTwoAssertsFail",2,"Expected: 5","ImaginaryPart","Message from Assert.Fail")
Multiple failures or warnings in test:
  1)   RealPart
  Expected: 5.0d
  But was:  5.2000000000000002d

  2)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  3) Message from Assert.Fail
  4)   StackTrace
  Expected: not null and String containing "MethodCallsFailAfterTwoAssertsFail"
  But was:  <string.Empty>

17) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("TwoAssertsFailAfterWarning",2,"WARNING","Expected: 5","ImaginaryPart")
Multiple failures or warnings in test:
  1) WARNING
  2)   RealPart
  Expected: 5.0d
  But was:  5.2000000000000002d

  3)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  4)   StackTrace
  Expected: not null and String containing "TwoAssertsFailAfterWarning"
  But was:  <string.Empty>

18) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("WarningAfterTwoAssertsFail",2,"Expected: 5","ImaginaryPart","WARNING")
Multiple failures or warnings in test:
  1)   RealPart
  Expected: 5.0d
  But was:  5.2000000000000002d

  2)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  3) WARNING
  4)   StackTrace
  Expected: not null and String containing "WarningAfterTwoAssertsFail"
  But was:  <string.Empty>

19) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("TwoAsserts_BothAssertsFail_Async",2,"RealPart","ImaginaryPart")
Multiple failures or warnings in test:
  1)   RealPart
  Expected: 5.0d
  But was:  5.2000000000000002d

  2)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  3)   StackTrace
  Expected: not null and String containing "TwoAsserts_BothAssertsFail_Async"
  But was:  <string.Empty>

20) Failed : NUnit.Framework.Assertions.AssertMultipleTests.AssertMultipleFails("TwoNestedBlocks_TwoAssertsFail_Async",3,"Expected: 5","ImaginaryPart")
Multiple failures or warnings in test:
  1)   Expected: 5
  But was:  4

  2)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

  3)   StackTrace
  Expected: not null and String containing "TwoNestedBlocks_TwoAssertsFail_Async"
  But was:  <string.Empty>

21) Failed : NUnit.Framework.TestContextTests.TestContextStoresFailureInfoForTearDown
  Expected: String containing "NUnit.TestData.TestContextData.TestTestContextInTearDown.FailingTest"
  But was:  <string.Empty>

22) Failed : NUnit.Framework.Internal.ReflectTests.DynamicInvokeWithTransparentExceptionsPreservesStackTrace
  Expected: property StackTrace String containing "MethodThrowingTargetInvocationException"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0
--- End of stack trace from previous location where exception was thrown ---

  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0
--- End of stack trace from previous location where exception was thrown ---

  at NUnit.Framework.Internal.ExceptionHelper.Rethrow (System.Exception exception) [0x00007] in <0a29a5784ee940e3ad5246f4d41be0cf>:0
  at NUnit.Framework.Internal.Reflect.DynamicInvokeWithTransparentExceptions (System.Delegate delegate) [0x00019] in <0a29a5784ee940e3ad5246f4d41be0cf>:0
  at NUnit.Framework.Internal.ExceptionHelper.RecordException (System.Delegate parameterlessDelegate, System.String parameterName) [0x00078] in <0a29a5784ee940e3ad5246f4d41be0cf>:0 "
  at NUnit.Framework.Internal.ReflectTests.DynamicInvokeWithTransparentExceptionsPreservesStackTrace () [0x00000] in <7b2e67d9e0c74508bd73a23d431def03>:0

23) Failed : NUnit.Framework.Internal.ReflectTests.InvokeWithTransparentExceptionsPreservesStackTrace
  Expected: property StackTrace String containing "MethodThrowingTargetInvocationException"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0
--- End of stack trace from previous location where exception was thrown ---

  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0
--- End of stack trace from previous location where exception was thrown ---

  at NUnit.Framework.Internal.ExceptionHelper.Rethrow (System.Exception exception) [0x00007] in <0a29a5784ee940e3ad5246f4d41be0cf>:0
  at NUnit.Framework.Internal.Reflect.DynamicInvokeWithTransparentExceptions (System.Delegate delegate) [0x00019] in <0a29a5784ee940e3ad5246f4d41be0cf>:0
  at NUnit.Framework.Internal.ExceptionHelper.RecordException (System.Delegate parameterlessDelegate, System.String parameterName) [0x00078] in <0a29a5784ee940e3ad5246f4d41be0cf>:0 "
  at NUnit.Framework.Internal.ReflectTests.InvokeWithTransparentExceptionsPreservesStackTrace () [0x00000] in <7b2e67d9e0c74508bd73a23d431def03>:0

24) Failed : NUnit.Framework.Internal.SetUpTearDownTests.HandleExceptionInBothSetUpAndTearDown
  Expected: String containing "NUnit.TestData.SetUpData.SetupAndTearDownExceptionFixture"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0
--TearDown
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

25) Failed : NUnit.Framework.Internal.SetUpTearDownTests.HandleExceptionInSetUp
  Expected: String containing "NUnit.TestData.SetUpData.SetupAndTearDownExceptionFixture"
  But was:  "  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

26) Failed : NUnit.Framework.Internal.SetUpTearDownTests.HandleExceptionInTearDown
  Expected: String containing "NUnit.TestData.SetUpData.SetupAndTearDownExceptionFixture"
  But was:  "--TearDown
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <6649516e5b3542319fb262b421af0adb>:0 "

[oznetmaster]

I had installed mono-runtime. Installing mono-complete did solve the missing dll problem.

I do not really need mono-complete, however :(

[oznetmaster]

@jnm2 There is no mono release 5.11 :(

Mono 5.x

    Mono 6.0.0 (not released yet)

    Mono 5.20.0 (11 Apr 2019)

    Mono 5.18.0 (21 Dec 2018)

    Mono 5.16.0 (08 Oct 2018)

    Mono 5.14.0 (07 Aug 2018)

    Mono 5.12.0 (08 May 2018)

    Mono 5.8.1 (26 Mar 2018)

    Mono 5.10.0 (26 Feb 2018)

    Mono 5.8.0 (01 Feb 2018)

    Mono 5.4.1 (31 Oct 2017)

    Mono 5.4.0 (05 Oct 2017)

    Mono 5.2.0 (14 Aug 2017)

    Mono 5.0.1 (23 May 2017)

    Mono 5.0.0 (10 May 2017)

[oznetmaster]

It really surprises me that we are running the integration tests under a mono release which is over a year old!

There have been four releases of mono since then.

I would have thought that the problems with the newer versions would have been identified and fixed by now :(

[jnm2]

@oznetmaster I have no idea what I was remembering. The last version of Mono that works for us is 5.18, and 5.20 introduces all those issues you mention.

It really surprises me that we are running the integration tests under a mono release which is over a year old!

This is what happens when we pin the Mono version in Travis due to needing to force a newer version.
Our Azure Pipelines build is constantly kept closer up to date. 18 days ago it was at 5.18.1, and now it's at 5.20.1: https://github.com/Microsoft/azure-pipelines-image-generation/blob/master/images/linux/Ubuntu1604-README.md

So the failures should start rolling in at any time from our AzDO build.

I would have thought that the problems with the newer versions would have been identified and fixed by now :(

Based on the dates you posted, the earliest we could have detected this incoming situation was ~5 weeks ago. Some of these failures are due to overspecified tests (stack traces), and others look like Mono must be plain broken because its behavior is not matching .NET Core or .NET Framework:

  2)   ImaginaryPart
  Expected: 4.2000000000000002d
  But was:  3.8999999999999999d

Mono had me on a fine chase recently. ThreadAbortExceptions disappeared, waited until the next unrelated exception is thrown, and came back to life with all the original context missing. The cause of this was a fundamental runtime spec violation. We saw the side effects of this ten years ago in a number of tests but couldn't figure out what was behind it, so we just disabled those tests on Mono.

I'm flattered by your faith in us that we're able to keep up with whatever Mono throws at us next, but it's also a time sink. Other than CI, it's hard for the team to even be aware of this because it's not practical for us to use Ubuntu when working on NUnit on a regular basis. (This thread abort bug for example afflicts only the non-Windows builds of Mono.)

[jnm2]

I do not really need mono-complete, however :(

Yeah, sounds like mono-devel might contain System.Web.

[oznetmaster]

Since I only maintain the CF version of NUnit, which does not involve CI at all, I have not been exposed to all these issues.

Now, I only need a reliable NUnit that works on the current Mono (5.20.1) on Ubunto 16.0.2.

I would be happy to help find the problems, but my environment is Mono on a different computer then I am running VS 2019 on, so debugging is very difficult.

[oznetmaster]

I do not control the mono version on that machine either.

[jnm2]

Thanks. It's probably down to you and me to sort these out. I will start working on it absolutely next (I just fixed #3258.) That means probably this weekend.

[oznetmaster]

Ok. Let me know what I can do.

[oznetmaster]

oznetmaster commented 17 seconds from now

Is this time travel on github now?