Assertion when the process runs out of memory #10415
Comments
benjaminknox
changed the title
|
Thanks for your report! In order to help us to identify the issue, could you:
if you enable this, it should provide better stack traces for managed frames.
does it happen everytime you start the program? After what period of time do you see the crash?
would be kinda interesting. you can use gist.github.com instead and post the link here. |
|
@lewurm Thank you for getting back to me, I cannot upload the EXE because it is closed source and owned by my employer. I will get everything else and post it here. I have been wondering if the fact that I compiled the EXE on windows with visual studio and simply uploaded it to our CentOS server to run would have anything to do with it? |
Unfortunately the EXE is proprietary
This happens every time, no regular interval. I noticed sometimes the program doesn't leave a stack trace. No pattern has yet been found.
I don't have that particular pid from the stack trace above, and I am also unsure if this is what you wanted, but here is a recent map from that folder: https://gist.github.com/benjaminknox/fd7859c034c54fd4879670ae87898aa4 Let me know if you mean something else. |
|
I also got exact error. However, my application is distributed, which is hard to reproduce. But in my observation, this error happen when I try to copy (or clone) a very large array (2^23 elements) to another array many times, continuously, and very stress. Mono version: 5.14.0.177 |
|
Yes, I have observed that when we decreased the amount of memory we were using it started happening less. |
|
potentially related: #10899 we shouldn't assert when |
…w guard page fails Related: mono#10415
|
@benjaminknox @vdkhoi could you test your application with this patch? #11064 |
|
Thank you @lewurm, will let you know |
|
I experience the same or a closely related issue with both, mono v5.16 and the current mono-git v5.21 (compiled just earlier today). When running certain parts of MaxQuant mass-spectrometry software, mono repeatedly crashes at about the same point in program execution: or Our system: I created a gist with the output of 3 runs with varying versions of mono and MaxQuant. The third file uses the latest mono-git from earlier today. Observations:
Let me know if you need more data. Unfortunately, MaxQuant is closed source and decompiling and dissecting is not an option for me at the moment. |
|
Looks like there is not much to do on the runtime side. The OS refuses to map more memory for the process. If you are confident that the system has more than enough memory available, one common issue that I've encountered on mono + huge memory scenarios is the process exceeding the maximum number of mappings limit. You can tweak that with And yes, if the process runs out of memory, we can crash, instead of throwing a graceful exception. |
|
Well, from a user perspective, a friendly |
|
We haven't been experiencing this issue anymore, it went away after we reduced the amount of memory our code was using. |
|
I'll close this issue now, if you have any further questions feel free to reopen. Thanks! |
marek-safar
changed the title
Steps to Reproduce
Current Behavior
Currently, the program runs for a while then crashes giving the above
Expected Behavior
The program doesn't crash and continues to run
On which platforms did you notice this
[ ] macOS
[ x ] Linux
[ ] Windows
Version Used:
Mono JIT compiler version 5.14.0.177 (tarball Sat Aug 4 08:32:47 UTC 2018)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS: __thread
SIGSEGV: altstack
Notifications: epoll
Architecture: amd64
Disabled: none
Misc: softdebug
Interpreter: yes
LLVM: yes(3.6.0svn-mono-/)
GC: sgen (concurrent by default)
Stacktrace
The text was updated successfully, but these errors were encountered: