Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

An exception occurred in the driver while loading tests. #270

Closed
goktugy opened this issue Jul 24, 2017 · 23 comments
Closed

An exception occurred in the driver while loading tests. #270

goktugy opened this issue Jul 24, 2017 · 23 comments
Labels

Comments

@goktugy
Copy link

goktugy commented Jul 24, 2017

I have the following setup:

ubuntu@ip-172-25-250-228:~$ ./NUnit.ConsoleRunner.3.7.0/tools/nunit3-console.exe SampleNUnit.dll -noresult
NUnit Console Runner 3.7.0
Copyright (c) 2017 Charlie Poole, Rob Prouse

Runtime Environment
OS Version: Linux 4.4.0.1020
CLR Version: 4.0.30319.17020

Test Files
SampleNUnit.dll

When I run the test I get the following:
Errors, Failures and Warnings

  1. Error :
    An exception occurred in the driver while loading tests.
    at (wrapper managed-to-native) System.Object:__icall_wrapper_mono_remoting_wrapper (intptr,intptr)
    at (wrapper remoting-invoke) NUnit.Engine.Agents.RemoteTestAgent:Run (NUnit.Engine.ITestEventListener,NUnit.Engine.TestFilter)
    at NUnit.Engine.Runners.ProcessRunner.RunTests (ITestEventListener listener, NUnit.Engine.TestFilter filter) <0x40700ce0 + 0x000ae> in :0

Test Run Summary
Overall result: Failed
Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2017-07-24 21:39:48Z
End time: 2017-07-24 21:39:48Z
Duration: 0.404 seconds

samplenunit

I would appreciate any help.

@goktugy
Copy link
Author

goktugy commented Jul 24, 2017

I have compiled my SampleNUNIT.dll with the following command:

mcs SampleNUnit.cs -target:library -r:NUnit.3.7.1/lib/net40/nunit.framework.dll -out:SampleNUnit.dll

@rprouse
Copy link
Member

rprouse commented Jul 24, 2017

What version of Mono do you have installed? We've had issues with several recent 4.x and 5.x releases, although different than that.

Can you also try running with the -inprocess flag to see if you get a different error? The callstack is in the remoting call.

Does it help if you run nunit3-console.exe with mono nunit3-console.exe?

@goktugy
Copy link
Author

goktugy commented Jul 24, 2017

I am running

Mono JIT compiler version 4.2.1 (Debian 4.2.1.102+dfsg2-7ubuntu4)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS: __thread
SIGSEGV: altstack
Notifications: epoll
Architecture: amd64
Disabled: none
Misc: softdebug
LLVM: supported, not enabled.
GC: sgen

The -inprocess flag returns: An exception occurred in the driver while loading tests.

Running "mono nunit3-console.exe" does not change the outcome. Any other recommendations?

@rprouse
Copy link
Member

rprouse commented Jul 25, 2017

Give me a bit. I am trying to repro on my Linux box, but MonoDevelop is refusing to add NuGet packages to my new project. As I said, Mono hasn't been very stable for us recently ☹️

@rprouse rprouse added Blocked and removed Blocked labels Jul 25, 2017
@rprouse
Copy link
Member

rprouse commented Jul 25, 2017

Compiling in a docker container the same way that you did give me a different but similar error using Mono 5.0.1.1. I think it is something to do with the way you are compiling. Is there a reason you aren't compiling the project, but just a single file?

NUnit Console Runner 3.7.0
Copyright (c) 2017 Charlie Poole, Rob Prouse

Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Runtime Environment
   OS Version: Linux 4.9.36.0
  CLR Version: 4.0.30319.42000

Test Files
    SampleNUnit.dll

Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8
Got a bad hardware address length for an AF_PACKET 16 8

Errors, Failures and Warnings

1) Error :
An exception occurred in the driver while loading tests.


Server stack trace:
  at NUnit.Engine.Runners.DirectTestRunner.LoadDriver (NUnit.Engine.Extensibility.IFrameworkDriver driver, System.String testFile, NUnit.Engine.TestPackage subPackage) [0x0003a] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0
  at NUnit.Engine.Runners.DirectTestRunner.LoadPackage () [0x000c4] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0
  at NUnit.Engine.Runners.TestDomainRunner.LoadPackage () [0x00017] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0
  at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded () [0x00008] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0
  at NUnit.Engine.Runners.DirectTestRunner.RunTests (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00000] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0
  at NUnit.Engine.Runners.AbstractTestRunner.Run (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00000] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0
  at NUnit.Engine.Agents.RemoteTestAgent.Run (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00000] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0
  at (wrapper managed-to-native) System.Runtime.Remoting.RemotingServices:InternalExecute (System.Reflection.MethodBase,object,object[],object[]&)
  at System.Runtime.Remoting.RemotingServices.InternalExecuteMessage (System.MarshalByRefObject target, System.Runtime.Remoting.Messaging.IMethodCallMessage reqMsg) [0x000b5] in <a07d6bf484a54da2861691df910339b1>:0

Exception rethrown at [0]:
  at (wrapper managed-to-native) System.Object:__icall_wrapper_mono_remoting_wrapper (intptr,intptr)
  at (wrapper remoting-invoke) NUnit.Engine.Agents.RemoteTestAgent:Run (NUnit.Engine.ITestEventListener,NUnit.Engine.TestFilter)
  at NUnit.Engine.Runners.ProcessRunner.RunTests (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00025] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0

@rprouse
Copy link
Member

rprouse commented Jul 25, 2017

If however I compile the solution and run the tests on that, it works fine. I expect that there are settings that need to be set for mcs that are present in the csproj file.

xbuild /p:Configuration=Release NUnitTest.sln

Then,

root@f624b67ceee8:/nunit# mono packages/NUnit.ConsoleRunner.3.7.0/tools/nunit3-console.exe NUnitTest/bin/Release/NUnitTest.dll
NUnit Console Runner 3.7.0
Copyright (c) 2017 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: Linux 4.9.36.0
  CLR Version: 4.0.30319.42000

Test Files
    NUnitTest/bin/Release/NUnitTest.dll

Run Settings
    DisposeRunners: True
    WorkDirectory: /nunit
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.5
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 2

Test Run Summary
  Overall result: Passed
  Test Count: 1, Passed: 1, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2017-07-25 01:02:28Z
    End time: 2017-07-25 01:02:29Z
    Duration: 0.896 seconds

Results (nunit3) saved as TestResult.xml

@rprouse
Copy link
Member

rprouse commented Jul 25, 2017

I am going to close this issue as a question. If you need further help, just post here. We will see it.

@rprouse rprouse closed this as completed Jul 25, 2017
@rprouse rprouse added this to the Closed Without Action milestone Jul 25, 2017
@WolfgangFahl
Copy link

WolfgangFahl commented Oct 30, 2017

Please reopen this. As outlined in https://stackoverflow.com/questions/45291234/running-nunit-tests-on-ubuntu-an-exception-occurred-in-the-driver-while-loading there should be no reason not to use mcs. I am using Mono JIT compiler version 5.0.1.1 and get the same weird error message which does not make any sense to me.

It looks like the issue is comparable to the situation a while ago at: https://stackoverflow.com/questions/12393425/on-osx-with-monodevelop-run-nunit-tests-from-terminal-command-line

It would be great to have a working command line that does not use visual studio, xamarin or mono develop.

@ChrisMaddock
Copy link
Member

@WolfgangFahl - please try using the console from this PR: #304 - you can either build the source, or find a package here: https://ci.appveyor.com/project/CharliePoole/nunit-console/build/3.8.0-ci-03798-issue-228b/artifacts

This won't solve the problem, but should give you a more in depth error message so you can diagnose better.

@CharliePoole
Copy link
Collaborator

I agree that it should be possible to build with simple Mono commands on Linux, without using xamarin or monodevelop. Some of our discussion of Linux support has become derailed - at least in my view - by focusing on how "escapees" from the Microsoft world typically build when working on Linux. But long-time users of Mono on linux typically use a command-line to build one file at a time and then link the object files together, generally as part of a script that provides a functional equivalent to a project file.

We could tell those folks we don't support the old methods, of course, but since distros like Debian that require building the packages they distribute from source actually build NUnit itself in that way, it's really in our interest that it should continue to work.

As the @nunit/core-team members already know, I plan to work separately on Linux support after my transition out of the NUnit team is complete.

@WolfgangFahl
Copy link

@ChrisMaddock and @CharliePoole
Thanks for looking into this so quickly.
At https://github.com/WolfgangFahl/fritz-csharp-api you can find what i tried to build and test.
I ran
./build
and then
./test

with the result below.

Unfortunately this project is not in my focus right now and my resources for it are too limited to follow a side track like this. I had hoped that with just a few lines of code things would work.

The build script at this time is:

#!/bin/bash
# WF 2017-10-30
#
# uncomment the following lines once
nuget install Nunit
nuget install NUnit.Runners
mcs /reference:System.Xml.Linq.dll /reference:System.Windows.Forms.dll fritz.cs
mcs fritztest.cs -target:library /reference:Fritz.exe /reference:NUnit.3.8.1/lib/net45/nunit.framework.dll -out:FritzTest.dll

and the test script ist:

#!/bin/bash
# WF 2017-10-30
ncr=NUnit.ConsoleRunner.3.7.0
export MONO_PATH=$(PWD)/$ncr/tools/:
mono $ncr/tools/nunit3-console.exe FritzTest.dll -noresult

I tried to use a fritztest.csproj file and msbuild but that format is far more complex than the few lines above.

E.g. i am trying to add an NUnit reference according to
https://stackoverflow.com/questions/16578819/understanding-a-csproj-assembly-reference
and
https://stackoverflow.com/questions/33824789/nunit-console-what-is-expected-in-csproj-file
and get all kind of error message but no working build.
The whole show seems to be a very moving target and non of the documentation I read in the last few hours seems to be up-to-date. For the time being I am leaving things in this frustrating state.

mono version
$mono --version

Mono JIT compiler version 5.0.1.1 (2017-02/5077205 Thu May 18 16:11:37 EDT 2017)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           normal
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  x86
	Disabled:      none
	Misc:          softdebug 
	LLVM:          yes(3.6.0svn-mono-master/8b1520c)
	GC:            sgen (concurrent by default)

output

./build 
Feeds used:
  https://api.nuget.org/v3/index.json

  GET https://api.nuget.org/v3/registration3-gz-semver2/nunit/index.json
  OK https://api.nuget.org/v3/registration3-gz-semver2/nunit/index.json 596ms


Attempting to gather dependency information for package 'Nunit.3.8.1' with respect to project '/Users/wf/source/csharp/fritz-csharp-api', targeting 'Any,Version=v0.0'
Gathering dependency information took 18,64 ms
Attempting to resolve dependencies for package 'Nunit.3.8.1' with DependencyBehavior 'Lowest'
Resolving dependency information took 0 ms
Resolving actions to install package 'Nunit.3.8.1'
Resolved actions to install package 'Nunit.3.8.1'
Retrieving package 'NUnit 3.8.1' from 'nuget.org'.
  GET https://api.nuget.org/v3-flatcontainer/nunit/3.8.1/nunit.3.8.1.nupkg
  OK https://api.nuget.org/v3-flatcontainer/nunit/3.8.1/nunit.3.8.1.nupkg 15ms
Adding package 'NUnit.3.8.1' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Added package 'NUnit.3.8.1' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Successfully installed 'NUnit 3.8.1' to /Users/wf/source/csharp/fritz-csharp-api
Executing nuget actions took 499,23 ms
Feeds used:
  https://api.nuget.org/v3/index.json

  GET https://api.nuget.org/v3/registration3-gz-semver2/nunit.runners/index.json
  OK https://api.nuget.org/v3/registration3-gz-semver2/nunit.runners/index.json 599ms


Attempting to gather dependency information for package 'NUnit.Runners.3.7.0' with respect to project '/Users/wf/source/csharp/fritz-csharp-api', targeting 'Any,Version=v0.0'
Gathering dependency information took 2,41 sec
Attempting to resolve dependencies for package 'NUnit.Runners.3.7.0' with DependencyBehavior 'Lowest'
Resolving dependency information took 0 ms
Resolving actions to install package 'NUnit.Runners.3.7.0'
Resolved actions to install package 'NUnit.Runners.3.7.0'
Retrieving package 'NUnit.ConsoleRunner 3.7.0' from 'nuget.org'.
Retrieving package 'NUnit.Extension.NUnitProjectLoader 3.5.0' from 'nuget.org'.
Retrieving package 'NUnit.Extension.NUnitV2Driver 3.6.0' from 'nuget.org'.
Retrieving package 'NUnit.Extension.NUnitV2ResultWriter 3.5.0' from 'nuget.org'.
Retrieving package 'NUnit.Extension.TeamCityEventListener 1.0.2' from 'nuget.org'.
Retrieving package 'NUnit.Extension.VSProjectLoader 3.5.0' from 'nuget.org'.
Retrieving package 'NUnit.Runners 3.7.0' from 'nuget.org'.
  GET https://api.nuget.org/v3-flatcontainer/nunit.consolerunner/3.7.0/nunit.consolerunner.3.7.0.nupkg
  OK https://api.nuget.org/v3-flatcontainer/nunit.consolerunner/3.7.0/nunit.consolerunner.3.7.0.nupkg 11ms
  GET https://api.nuget.org/v3-flatcontainer/nunit.extension.nunitv2resultwriter/3.5.0/nunit.extension.nunitv2resultwriter.3.5.0.nupkg
  OK https://api.nuget.org/v3-flatcontainer/nunit.extension.nunitv2resultwriter/3.5.0/nunit.extension.nunitv2resultwriter.3.5.0.nupkg 11ms
  GET https://api.nuget.org/v3-flatcontainer/nunit.runners/3.7.0/nunit.runners.3.7.0.nupkg
  OK https://api.nuget.org/v3-flatcontainer/nunit.runners/3.7.0/nunit.runners.3.7.0.nupkg 13ms
Adding package 'NUnit.ConsoleRunner.3.7.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
  GET https://api.nuget.org/v3-flatcontainer/nunit.extension.nunitprojectloader/3.5.0/nunit.extension.nunitprojectloader.3.5.0.nupkg
  OK https://api.nuget.org/v3-flatcontainer/nunit.extension.nunitprojectloader/3.5.0/nunit.extension.nunitprojectloader.3.5.0.nupkg 24ms
Added package 'NUnit.ConsoleRunner.3.7.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
  GET https://api.nuget.org/v3-flatcontainer/nunit.extension.vsprojectloader/3.5.0/nunit.extension.vsprojectloader.3.5.0.nupkg
Successfully installed 'NUnit.ConsoleRunner 3.7.0' to /Users/wf/source/csharp/fritz-csharp-api
Adding package 'NUnit.Extension.NUnitProjectLoader.3.5.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Added package 'NUnit.Extension.NUnitProjectLoader.3.5.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Successfully installed 'NUnit.Extension.NUnitProjectLoader 3.5.0' to /Users/wf/source/csharp/fritz-csharp-api
  OK https://api.nuget.org/v3-flatcontainer/nunit.extension.vsprojectloader/3.5.0/nunit.extension.vsprojectloader.3.5.0.nupkg 11ms
  GET https://api.nuget.org/v3-flatcontainer/nunit.extension.teamcityeventlistener/1.0.2/nunit.extension.teamcityeventlistener.1.0.2.nupkg
  OK https://api.nuget.org/v3-flatcontainer/nunit.extension.teamcityeventlistener/1.0.2/nunit.extension.teamcityeventlistener.1.0.2.nupkg 11ms
  GET https://api.nuget.org/v3-flatcontainer/nunit.extension.nunitv2driver/3.6.0/nunit.extension.nunitv2driver.3.6.0.nupkg
  OK https://api.nuget.org/v3-flatcontainer/nunit.extension.nunitv2driver/3.6.0/nunit.extension.nunitv2driver.3.6.0.nupkg 12ms
Adding package 'NUnit.Extension.NUnitV2Driver.3.6.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Added package 'NUnit.Extension.NUnitV2Driver.3.6.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Successfully installed 'NUnit.Extension.NUnitV2Driver 3.6.0' to /Users/wf/source/csharp/fritz-csharp-api
Adding package 'NUnit.Extension.NUnitV2ResultWriter.3.5.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Added package 'NUnit.Extension.NUnitV2ResultWriter.3.5.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Successfully installed 'NUnit.Extension.NUnitV2ResultWriter 3.5.0' to /Users/wf/source/csharp/fritz-csharp-api
Adding package 'NUnit.Extension.TeamCityEventListener.1.0.2' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Added package 'NUnit.Extension.TeamCityEventListener.1.0.2' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Successfully installed 'NUnit.Extension.TeamCityEventListener 1.0.2' to /Users/wf/source/csharp/fritz-csharp-api
Adding package 'NUnit.Extension.VSProjectLoader.3.5.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Added package 'NUnit.Extension.VSProjectLoader.3.5.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Successfully installed 'NUnit.Extension.VSProjectLoader 3.5.0' to /Users/wf/source/csharp/fritz-csharp-api
Adding package 'NUnit.Runners.3.7.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Added package 'NUnit.Runners.3.7.0' to folder '/Users/wf/source/csharp/fritz-csharp-api'
Successfully installed 'NUnit.Runners 3.7.0' to /Users/wf/source/csharp/fritz-csharp-api
Executing nuget actions took 265,68 ms
pan:fritz-csharp-api wf$./test
NUnit Console Runner 3.7.0 
Copyright (c) 2017 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: MacOSX 16.6.0.0 
  CLR Version: 4.0.30319.42000

Test Files
    FritzTest.dll


Errors, Failures and Warnings

1) Error : 
An exception occurred in the driver while loading tests.


Server stack trace: 
  at NUnit.Engine.Runners.DirectTestRunner.LoadDriver (NUnit.Engine.Extensibility.IFrameworkDriver driver, System.String testFile, NUnit.Engine.TestPackage subPackage) [0x0003a] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0 
  at NUnit.Engine.Runners.DirectTestRunner.LoadPackage () [0x000c4] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0 
  at NUnit.Engine.Runners.TestDomainRunner.LoadPackage () [0x00017] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0 
  at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded () [0x00008] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0 
  at NUnit.Engine.Runners.DirectTestRunner.RunTests (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00000] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0 
  at NUnit.Engine.Runners.AbstractTestRunner.Run (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00000] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0 
  at NUnit.Engine.Agents.RemoteTestAgent.Run (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00000] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0 
  at (wrapper managed-to-native) System.Runtime.Remoting.RemotingServices:InternalExecute (System.Reflection.MethodBase,object,object[],object[]&)
  at System.Runtime.Remoting.RemotingServices.InternalExecuteMessage (System.MarshalByRefObject target, System.Runtime.Remoting.Messaging.IMethodCallMessage reqMsg) [0x000b5] in <ffb99659fc1c47faa4d5f883014d08aa>:0 

Exception rethrown at [0]: 
  at (wrapper managed-to-native) System.Object:__icall_wrapper_mono_remoting_wrapper (intptr,intptr)
  at (wrapper remoting-invoke) NUnit.Engine.Agents.RemoteTestAgent:Run (NUnit.Engine.ITestEventListener,NUnit.Engine.TestFilter)
  at NUnit.Engine.Runners.ProcessRunner.RunTests (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00025] in <1e8ad6af4c6f4686ad7e5f9e67020b3b>:0 

Test Run Summary
  Overall result: Failed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2017-10-30 16:04:45Z
    End time: 2017-10-30 16:04:45Z
    Duration: 0.614 seconds

@WolfgangFahl
Copy link

I have worked around the issue with https://github.com/WolfgangFahl/fritz-csharp-api/blob/master/fritzsimpletest.cs - i now feel foolish since i only needed to show a few comparisons and this is easily done with a single checkMD5 function.

@davidenq
Copy link

davidenq commented Jan 15, 2018

I've spent several days trying to solve this issue as well, but only there isn't enough information how to use Mono and NUnit using only command line on Linux. So, I've tested in Ubuntu 17.10 and Docker with Alpine 3.7 and I've used some extra environment vars to specify in export MONO_PATH, where mono should find nunit.framework.dll and classes that I want to test. and finally all tests were executed successful!.

this post was very useful to find a solution. I hope this information is useful.

Example

  1. I've used only a script through command line to compile and execute tests without any IDE.
  2. I've installed mono via sudo apt-get mono for Ubuntu and apk add mono-dev@testing for Alpine
  3. I've downloaded NUnit and NUnit.Runners using:
$ mono nuget.exe install NUnit -Version 3.9.0 -OutputDirectory "/home/davidenq/csharp/vendor/" 
$ mono nuget.exe install NUnit.Runners -Version 3.7.0 -OutputDirectory "/home/davidenq/csharp/vendor/" 

folder structure

├── ~/
│   │   ├── csharp/
│   │   │   ├── classes/
│   │   │   │   ├── HelloWorld.cs
│   │   │   ├── tests/
│   │   │   │   ├── HelloWorldTest.cs
│   │   │   ├── vendor/
│   │   │   │   ├── NUnit.3.9.0/
│   │   │   │   │   ├── lib/
│   │   │   │   │   │   ├── net45/
│   │   │   │   ├── NUnit.ConsoleRunner.3.7.0/
│   │   │   │   │   ├── tools/
│   │   │   │   ├── NUnit.Extension.NUnitProjectLoader.3.5.0/
│   │   │   │   ├── NUnit.Extension.NUnitV2Driver.3.6.0/
│   │   │   │   ├── NUnit.Extension.NUnitV2ResultWriter.3.5.0/
│   │   │   │   ├── NUnit.Extension.TeamCityEventListener.1.0.2/
│   │   │   │   ├── NUnit.Extension.VSProjectLoader.3.5.0/
│   │   │   │   ├── NUnit.Runners.3.7.0/
|   ├── script.sh

script.sh

#!/bin/bash

#Make sure the mono is finding those assemblies first
CLASSES=~/csharp/classes
TESTS=~/csharp/tests
TOOLS=~/csharp/vendor/NUnit.ConsoleRunner.3.7.0/tools
NET=~/csharp/vendor/NUnit.3.9.0/lib/net45

export MONO_PATH=$CLASSES:$TOOLS:$NET

$mcs $CLASSES/HelloWorld.cs -t:library
$mcs $TESTS/HelloWorldTest.cs -r:$NET/nunit.framework.dll,$CLASSES/HelloWorld.cs -t:library

mono $TOOLS/nunit3-console.exe HelloWorldTest.dll

execute script

./script.sh

output

NUnit Console Runner 3.7.0 
Copyright (c) 2017 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: Linux 4.13.0.25 
  CLR Version: 4.0.30319.42000

Test Files
    ~/csharp/tests/HelloWorld.dll


Run Settings
    DisposeRunners: True
    WorkDirectory: ~
    ImageRuntimeVersion: 4.0.30319
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 4

Test Run Summary
  Overall result: Passed
  Test Count: 2, Passed: 2, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2018-01-15 01:20:03Z
    End time: 2018-01-15 01:20:03Z
    Duration: 0.843 seconds

Extra information

Information about Mono, NUnit and NUnit.Runners on Ubuntu

Distributor ID: Ubuntu x86_64
Description:    Ubuntu 17.10
Codename:       artful
Process: Intel(R) Core(TM) i5-3330 CPU @ 3.00GHz
Memory:  8Gb DDR3

Mono

Mono JIT compiler version 5.4.1.6 (tarball Wed Nov  8 20:37:08 UTC 2017)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug 
        LLVM:          supported, not enabled.
        GC:            sgen (concurrent by default)

NUnit and NUnit.Runner

NUnit 3.9.0
NUnit Console Runner 3.7.0 
Copyright (c) 2017 Charlie Poole, Rob Prouse

Runtime Environment
   OS Version: Linux 4.13.0.25 
  CLR Version: 4.0.30319.42000

================

Information about Mono, NUnit and NUnit.Runner on Docker with Alpine using Docker Alpine glibc

mono
Mono JIT compiler version 5.2.0.224 (tarball Tue Sep 26 15:47:31 GMT 2017)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       normal
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen (concurrent by default)

@WolfgangFahl
Copy link

Please reopen this issue. It is a pitty if it is not tracked as such.

@rprouse
Copy link
Member

rprouse commented Jan 15, 2018

@WolfgangFahl I would be happy to re-open the issue if it is an NUnit problem and not a configuration problem. @davidenq managed to get it working above.

When you compile, is the referenced nunit.framework.dll assembly copied into the same directory as your test.dll? Are other referenced assemblies? If not, do the tests run if you manually copy the files in?

@rprouse
Copy link
Member

rprouse commented Jan 15, 2018

As for the debian compile from source issue, that is separate, this issue talks about compiling test assemblies from the command line, not compiling NUnit from the command line. Besides, Debian hasn't released a version of NUnit since 2.6.4, https://packages.debian.org/search?keywords=nunit&searchon=names&suite=stable&section=all. It isn't recent changes that caused that.

@davidenq
Copy link

@rprouse Actually, I have neither copied nor moved any *.dll assembly from its own folder to another directory. I've only referenced from my script to the nunit.framework.dll assembly and all my tests there have been executed without any error. See the above example (the script and folder structure), nothing is moved, it's only done references.

@rprouse
Copy link
Member

rprouse commented Jan 16, 2018

@davidenq if I am reading your issue right, you had to add environment variables so that nunit.framework is found, I believe that @WolfgangFahl's problem is that the framework is not in the probing path, so not found, causing his error.

@rprouse
Copy link
Member

rprouse commented Jan 16, 2018

Normally, when you compile a csproj file, any referenced assemblies are copied into the build directory unless you specify otherwise. I assumed that mcs would do the same when you specify references, but looking at the errors, that doesn't seem to be the case, so I am asking for confirmation. I don't have much experience for command line compiles using mcs and it may be that there is an option that causes referenced assemblies to be copied.

@davidenq
Copy link

I guess that those assemblies are not copied into the build directory. At least, I have not seen that that happen. But, I going to do new tests to verify what is the real behavior. I guess that it will be more useful share with you a dockerfile file preconfigured with Mono, NUnit and scripts to test. What do you think?

@rprouse
Copy link
Member

rprouse commented Jan 18, 2018

@davidenq the NUnit console needs to be able to find the framework DLL and load it to run your tests, so it needs to be in the probing path. The most common place for that is in the same directory as the test assembly. Building with MSBuild or XBuild performs that step for you, although it might be msc/csc that does it with appropriate command line options.

You can compile NUnit tests with msc or csc and they will run, but you need to get the command line parameters correct and you may also need to run other commands like nuget restore. I know this because xbuild and msbuild end up calling those commands under the covers. The command lines are extremely long though, so I do not have the time nor the interest in figuring them out or documenting them. Especially when there are tools to simplify the process.

If you want to figure this out and document it, I would be happy to change this to a documentation issue and turn it over to you. To me, it isn't much different than C++. I can compile and link individual CPP files, but would never do that. Even for smaller projects I would use a simple make file.

I compile at the command line all the time, but I would still rather type;

nuget restore .\NUnitCommandLine.sln
MSBuild.exe .\NUnitCommandLine.sln /t:build

than,

 C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\bin\Roslyn\csc.exe /noconfig /nowarn:1701,1702 /nostdlib+ /errorreport:prompt /warn:4 /define:DEBUG;TRACE /highentropyva+ /reference
  :"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\Microsoft.CSharp.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\mscorl
  ib.dll" /reference:C:\src\Spikes\packages\NUnit.3.9.0\lib\net45\nunit.framework.dll /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\System.Core.dll" /reference:"
  C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\System.Data.DataSetExtensions.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\
  v4.5\System.Data.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\System.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NE
  TFramework\v4.5\System.Net.Http.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\System.Xml.dll" /reference:"C:\Program Files (x86)\Reference Assemblies\Micr
  osoft\Framework\.NETFramework\v4.5\System.Xml.Linq.dll" /debug+ /debug:full /filealign:512 /optimize- /out:obj\Debug\NUnitCommandLine.dll /ruleset:"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterpri
  se\Team Tools\Static Analysis Tools\\Rule Sets\MinimumRecommendedRules.ruleset" /subsystemversion:6.00 /target:library /utf8output TestClass.cs Properties\AssemblyInfo.cs "C:\Users\rob.prouse\AppData\Local\Te
  mp\.NETFramework,Version=v4.5.AssemblyAttributes.cs"

Or the msc equivalent.

Even cleaning that up and removing unnecessary references leads to,

 C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\bin\Roslyn\csc.exe /noconfig /nowarn:1701,1702 /nostdlib+ /define:DEBUG;TRACE /highentropyva+ 
   /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\Microsoft.CSharp.dll" 
   /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\mscorlib.dll"
   /reference:C:\src\Spikes\packages\NUnit.3.9.0\lib\net45\nunit.framework.dll
   /reference:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5\System.dll"
   /debug+ /debug:full /filealign:512 /optimize- /out:obj\Debug\NUnitCommandLine.dll /subsystemversion:6.00 /target:library /utf8output TestClass.cs

@WolfgangFahl
Copy link

I think a preconfigured dockerfile that is maintained by the Nunit team would be a major asset.

@rprouse
Copy link
Member

rprouse commented Jan 19, 2018

@WolfgangFahl how would the dockerfile work? Wouldn't you still need to modify the build steps to include different test files, modify references, etc? Or are you talking about a dockerfile to build NUnit itself? We have a batch file in the framework that builds NUnit in a Linux container, we could do something similar here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants