NUnit-Runner 3.9.0-dev-03972 exit with error: "Exception encountered unloading application domain"

[ghost]

Hi Guys.
We were having the problem stated at #128, so we decided to upgrade our NUnit-Console from 3.5 to the latest version (3.8)

(I did some research and found the threads #191, #321, #371 and #403)

I'm having the problem stated at #191 where @Rajkumar-Uppala and @ChrisMaddock pointed that some warnings in NUnit-Console 3.5 turned into exceptions.

I decided to upgrade the version to 3.9 (https://www.myget.org/feed/nunit/package/nuget/NUnit.ConsoleRunner/3.9.0-dev-03972)
and I'm still facing this problem.
Here are some screen shots:

I have latest build from master at this folder

And I'm running the console at this folder to run the DLL

And despite the tests passing, I'm facing this error

Here is the full stack trace from the error:

NUnit.Engine.NUnitEngineUnloadException : Agent Process was terminated successfully after error.
----> NUnit.Engine.NUnitEngineUnloadException : Exception encountered unloading application domain
----> NUnit.Engine.NUnitEngineException : Exception encountered unloading application domain: Erro ao descarregar appdomain. (Exceção de HRESULT: 0x80131015)
Application domain name: domain-
Application domain BaseDirectory: C:\GoCD\Agent3\pipelines\Wms.Testes.Develop\output\Tests\Hbsis.Wms.Rastreabilidade_OLD.Testes
Error trying to read application domain details: Não foi possível carregar arquivo ou assembly 'websocket-sharp, Version=1.0.2.42043, Culture=neutral, PublicKeyToken=5660b08a1845a91e' ou uma de suas dependências. O sistema não pode encontrar o arquivo especificado.
em System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
em System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
em System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean forIntrospection)
em System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
em System.Reflection.Assembly.Load(String assemblyString)
em System.UnitySerializationHolder.GetRealObject(StreamingContext context)

em System.AppDomain.ReflectionOnlyGetAssemblies()
em NUnit.Engine.Internal.DomainDetailsBuilder.DetailsFor(AppDomain domain, String errMsg)

It looks like the error itself is not a problem with NUnit-Console, as stated, the "websocket-sharp" dll was not found.
The point is that running the same DLL with NUnit-Console v3.5 works fine, as we can see in the following prints:

My Issue is:
Running the DLLs with v 3.5 in our CI (GoCd works fine). The websocket problem is not an issue to us and I think it is not worth the time to look for that.
Running the DLLs with v 3.8 gives me an error in the CI and v 3.9 gives me the error on the console (did not try 3.9 in the CI, but I assume it will the same behavior

Here are the prints of the same dlls used in the above prints in our CI, but with the the v3.5:

Is it possible to turn this error back to a warning, as pointed in the issue #371 by @chillitom?

[frbatist]

+1

[ChrisMaddock]

Interesting. This exception should be caught and just display what the error was. See https://github.com/nunit/nunit-console/blob/master/src/NUnitEngine/nunit.engine/Internal/DomainDetailsBuilder.cs#L65

@hbsis-williamdasilva - can you run this in NUnit Console directly and check what exit code you get?

[ghost]

@ChrisMaddock on the print above the stack trace I'm running directly with NUnit-Console, but at the end I don't get the message of the error code (I remember that I saw in some print of yours in another issue where there was the code). Do I need some extra parameter to show the exit code?

[ChrisMaddock]

@hbsis-williamdasilva - see here: https://superuser.com/questions/123008/get-exit-status-of-last-executed-command-in-windows

[ghost]

Thanks @ChrisMaddock . I'll just wait my team finish an activity in our environment and I will run the DLL again to get the exit status

[ghost]

So

I'm receiving the exit code 0.

Could this be related to another issue then?

[chillitom]

What CI server/build technology are you using? Some will fail a build if they detect the word “error” in the std out or if anything writes to std err.

…

Il giorno 24 apr 2018, alle ore 20:49, hbsis-williamdasilva ***@***.***> ha scritto: So I'm receiving the exit code 0. Could this be related to another issue then? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[ChrisMaddock]

Yeah, exit code zero is what we'd expect.

It may be that CircleCI is reading the logs and thinking anything that prints a stack trace is a failure? I'm not sure there though - afraid I don't use CircleCI. @nunit/engine-team - I feel like we might have seen something like this before where CI systems detect the word 'error' in logging and assume failure - do we have a convention here?

@hbsis-williamdasilva - in my opinion currently, this is working as intended. I've submitted #411 however, to make the output a little less scary for what is a minor issue. I'd look into CircleCI and see if there's any way to avoid this behaviour, if it is the case.

Edit: Thanks @chillitom! Confirmed my suspicions!

[ghost]

@ChrisMaddock @chillitom
We use GoCD. I will check at the documentation if they fail the build checking for "error" in the log.

I agree that if 0 is the expected, then it is not an issue. But it is very strange that the build is failing because of that.

I will setup our environment in GoCD to see if it fails with v3.9 returning this log with "error".
I will provide all the info I gather tomorrow, both from GoCD and from the scenarios that I use.

Thanks for the help.

[ChrisMaddock]

We use GoCD

Sorry, short term memory problems! 😄

[ghost]

Hey @ChrisMaddock , I bring you good news:

So, after a long night of rest and thinkering, today I could make some tests and things are fine now.

Yesterday we test v 3.8 in our CI and the build failed. So I donwloded the v 3.9 and ran our dll with the NUnit-Console (not in the CI), and we got the same error on v 3.8.

After you showed me how to see the error code, we got the error code 0 when running NUnit Console with v 3.9. So today's morning I did the same thing with v 3.8 and got the error code -5.

Thing is that the error showed on the console was the same, who led us think that running v 3.8 and v 3.9 would get the same result.

So, I upgraded to v 3.9 on our CI environment and runned the build again:

I see #411 improved the message, wich I belive will help, 'cause now it doesn't looks like the end of the world after running the tests.,Maybe, if this could help, I would suggest, at the end of the stack trace, to print the Exit Code too.

So problem solved, v 3.9 works fine 😀
Thanks for the help and the time.
I hope this thread can help people who got the same situation.

[ChrisMaddock]

Great - glad you're sorted out!

[wvetrone]

We are running 3.9 and we still get that exception.
Results (nunit3) saved as C:\Jenkins\workspace\Test Orchestrator\TestResult-LA-FARM02-W10_1.xml

NUnit.Engine.NUnitEngineUnloadException : Exception encountered unloading application domain
----> NUnit.Engine.NUnitEngineException : Exception encountered unloading application domain: Error while unloading appdomain. (Exception from HRESULT: 0x80131015)
Application domain name: domain-
Application domain BaseDirectory: C:\test\test-framework\

[ChrisMaddock]

@wvetrone Please can you open a new issue on this, with you full console output and the logs generated by running with --trace=Debug. Thanks!

[hemrajpatil1]

I too face same issue. Created another ticket - #840