-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Exception encountered unloading AppDomain #191
Comments
I'm not sure this is related to the failures fixed in #168. Is this new for you since 3.6? More information should be written to the log in this case. Can you try to reproduce with @nunit/engine-team
From what I can find out CannotUnloadAppDomainException will be thrown if the user code can not be terminated immediately, e.g. if finalizers are running. Should we perhaps be retrying the unload over 30 seconds or so? There's a 30 second timeout, but that's only for a hang - whereas this would throw immediately and crash, as far as I can see? |
Yes 3.5 works fine. 3.6.0 had issue #168. With 3.6.1 it's this. |
The log doesn't contain much: |
I started getting the same error on our build chain recently. The commit that broke the build was a change to the nunit console parameters to set workers and agents to 1 to try and debug a concurrency issue in some of our tests. We've been running 3.6.1 successfully previously. I've bumped the tracing level to Verbose but don't get anything significant.
As a side note it's a bit of a shame to only log exception messages in the internal logs. Here we don't get any information at all since apparently the exception has no message. Logging exception.ToString() would be a lot more useful IMHO. |
There's a separate issue to improve the logging here at: #111 Doing that issue would probably be a help in working out what's going on here. |
I just ran into this with our integration test suite and I'm fairly certain it's because of the 30 second time out, would be handy to have a configurable override for that setting at least... |
I'm traveling at the moment but here's a quick update. I debugged the
process on the CI machine and found the culprit in our tests. One of them
was holding an active thread that was blocking the unloading.
I think a good option would be to dump the callstacks of all threads of the
agent processes before throwing the unload exception. That would help a lot
to find what is going on.
Cheers
Le 12 avr. 2017 1:51 AM, "Michael Letterle" <notifications@github.com> a
écrit :
… I just ran into this with our integration test suite and I'm fairly
certain it's because of the 30 second time out, would be handy to have a
configurable override for that setting at least...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#191 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABrccs7xVM02g5kFDOaHOwusk1ckZ1d7ks5rvBIfgaJpZM4MZOOh>
.
|
I also got this issue, but I'm not sure why, only information I've is this:
Did you found any workaround? |
I did not find a workaround but I found the reason the unloading was failing. And it was in our code. To find out what was blocking I logged on remotely to our build server, ran Visual Studio on it and debugged the NUnit processes directly on the machine. |
Follow up on this. We've had another similar issue that cropped up on our (rather big) build chain. This time the issue is in third party code we can't really fix. So I ended up forking NUnit to remove the unload exception altogether, as an experiment. Builds have been consistently successful since then. It might make sense to downgrade from a fatal exception to something milder? I definitely think it should bubble up so that users are aware of it but throwing means that no test report is generated at all. Maybe it should still write a report but mark it as failed with the CannotUnloadAppDomainException error message in the report? This way the tests results would be available but the error would be plainly visible. |
The trick here is that the tests have already been run and the result reported before this error actually occurs. Since the problem happens "between" test runs - in the case of the console after the sole test run - there's no place to report it except in an exception of some kind. So, it comes down to a matter of how the console handles the exception. NUnitEngineException is used to report errors in the engine, which usually mean the test cannot be run. Perhaps we should not be wrapping this exception but let the console handle the CannotUnloadAppDomainException and just issue a warning. |
NCrunch does this IIRC |
I have the same issue, after passing all tests, I get
Nunit Version: 3.7.0 |
Just fyi, we worked around this problem by fixing the root cause of the "unable to unload" problem. |
Thanks for the update @Trass3r - good to know your up and running. For this issue - I think @CharliePoole's point is worth investing some time into, this could suitably be handled as a warning by the console, and still allow the results to be written. |
Same error here on nunit3-console v3.7. I've removed PS. In my case downgrading to v3.5 did not work. |
@Fernell Funny thing, you've put us back to our traditional approach of ignoring these errors. We added an exception when people complained about the silent failure. We've been tinkering with it ever since. |
Well, you can not satisfy everyone. I was struggling with this error for some time because our code seems to work properly (all tests are passing when I commented this exception) and I was unable to find out why it crashes anyway. Perhaps it should be configurable if exception should be thrown or not (like some --silent-domain-unload-failure flag). Anyway sorry for being troublemaker here. |
No bother... I preferred the old approach myself. However, it causes a problem when a large number of appdomains can't be unloaded and continue to take up memory. They may also be holding other resources that are needed by the tests. Personally, I would favor removing the exception in order to design some better way to notify the user of the problem. |
100% agree too. IMHO, it's better to maybe loose a notification than just prevent the whole tests to execute. |
This suggestion of yours is still the best idea imo @CharliePoole. Just needs someone to do a pull request for it... @Fernell or @jgrossrieder, either of you interested? 😄 |
Hi Team, We are facing an issue with "Exception encountered unloading AppDomain" then it is terminating with "Agent Process was terminated successfully after error" . Could anyone of please advise me how to overcome this. Any advise on this would be greatly helpful for to proceed with upgradation of Nunit to our Projects. [14:28:13][Step 8/9] Errors, Failures and Warnings Thanks & Regards, |
@ChrisMaddock - thank you for your advise on this Could you please help me to elaborate how to do this. Please advise Thanks & Regards, |
@Rajkumar-Uppala Sorry, I'm not a TeamCity user myself, but see 'debugging nunit tests' on this page. |
You'll want to pass that .nunit file into the NUnit Console - as TeamCity is doing. Visual Studio shouldn't come into it. If you're currently running in the visual studio test window, you won't see this exception due to the way tests are run. Hopefully running in the console may give you a repro that you can they debug. |
@ChrisMaddock - Thank you for the advise. |
Hi Everyone, With the implementation of mock service instead of referring to actual service solves the issue. Hope my investigation will help someone to investigate their issues considering this corner also. Thanks & Regards, |
@Rajkumar-Uppala - thanks for following up. You're correct this has changed from a warning to an error - as you'll see above, we're actually considering changing it back! In my opinion, it's important to flag it to the user in a non-ignorable way, so that the user can make some attempt to fix the issue. As you've seen - it often highlights a more fundamental issue in user-code. It would be better if we could write the results file out however, and just exit with a non-zero code. |
@ChrisMaddock - thanks for your comments on my reply. Please try to implement a process to include more information in detail as part of release notes/breaking changes (If there is any possibility) by which everyone will be benefited.
Hope you understand my concern behind raising this. |
It's a bit difficult to suggest remediation for a problem in user code! 🙂 As you say, the cause of your fail-to-unload is likely different to everyone else's cause in this thread! We recognise it's currently not ideal though - so what we're doing to improve this experience:
NUnit is open source software however - all contributions welcome! So if you have any concrete ideas on how we can improve things, and the team agrees with your suggestion, please share them and we can improve things for everyone! 😄 |
@ChrisMaddock - I accept and can understand difficulty or pain to implement my suggestion. Thank you everyone in the thread for sharing your best thoughts against the issues raised here .. for me @ChrisMaddock gives his best.... to address this thread issues |
Been taking a look at this today. A key thing I don't think we've been aware of so far - results only seem to be lost when using an I'll loop round to look at that problem later. |
@ChrisMaddock wow, #116 is old! I was so unfamiliar with everything back then. Would you like to take it since I have other stuff going on? 😁 😁 |
Please! Was just having more of a dig - I'm not even totally sure they are the same bug right now, I think it may just be that #116 causes a specific case to be exposed which would be a bit more niche otherwise. 😄 |
Ah, it is. This can be repro'd with:
#116 is only involved, because that's effectively causing the NUnit Project loader to do the above. Two separate bugs really. 😄 |
#116 is dead last on my NUnit todo list, just so you know. 😁 |
No worries - I'll see how I get on! #116 I'm less worried about, nobodies really been chasing that one. I wanted to get a fix in for this issue, as it keeps attracting people at the moment. 😄 |
Hi Guys. I'm still having some issues related to this thread in version 3.8 of Nunit Console. We just upgraded from version 3.5 to 3.8. The issue is that it looks like it can't find the dll "websocket-sharp" but that does not change the result of our tests I saw the solution #321 and some other related threads, but could not find an option to turn this exception back to an warning. Quoting @CharliePoole " I would favor removing the exception in order to design some better way to notify the user of the problem.", could you guys please tell me if that was implemented? Is there any way to change this exception back to a warning? At the moment, I'm trying to fix the websocket-sharp error, but if there is an option to turn the exception back to a warning it would be much better, since this kind of errors are not an issue to us |
@hbsis-williamdasilva - See #403. Current solution is to use the build of master from MyGet (see details in README) - this will revert to existing zero in the next release. |
@ChrisMaddock Thanks for the reply. And got the nunit3-console.exe I'm using this console to run the dll that I got the error with v 3.8, as we can see here But I'm still getting the error Am I missing something? |
@hbsis-williamdasilva Ah sorry, I misread - I believe you're seeing a separate exception thrown when trying to print app domain details. Would you file a new issue for this? Apologies! |
Sure, thanks for the help. I'll create a new issue |
@ChrisMaddock Issue created #410 |
We tested the fix for #168 via the 3.6.1 NuGet release but still get other sporadic failures:
nunit3-console.exe ..... --work=.../Release --out=TestOutput.log --result=TestResult.xml --labels=On --framework=net-4.5 --dispose-runners --agents=8
The text was updated successfully, but these errors were encountered: