Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Message "Operating system call munmap failed. Error code 12" can appear in firebird.log under heavy load (2.5.x, CS, SC) [CORE5643] #5909

Closed
firebird-issue-importer opened this issue Oct 19, 2017 · 6 comments

Comments

@firebird-issue-importer

Submitted by: @pavel-zotov

Alex asked me to open special ticket for this issue.

Host has several databases which are served by single FB instance. Total number of attachments in work time is ~550.
After some time of normal work following message "Operating system call munmap failed. Error code 12" - did appear in firebird.log four times, and after that fb_smp_server was crashed.
I've sent stack-trace and additional info by e-mail, 18.10.2017 23:48.

Commits: d9f53fa f5a37fd a65276b

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Oct 19, 2017

Modified by: @AlexPeshkoff

assignee: Alexander Peshkov [ alexpeshkoff ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Oct 25, 2017

Modified by: @AlexPeshkoff

summary: Message "Operating system call munmap failed. Error code 12" can apper in firebird.log under heavy load (2.5.x, CS, SC) => Message "Operating system call munmap failed. Error code 12" can appear in firebird.log under heavy load (2.5.x, CS, SC)

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Oct 31, 2017

Commented by: @AlexPeshkoff

Certainly one can't change Linux behavior regarding splitting VMA (Virtual Memory Area) structures in munmap() call.
What we get with this fix - memory block which can't be unmapped due to ENOMEM error is saved in linked list in MemoryPool and reused as soon as possible. I.e. we avoid throwing exception (which often happens in dtors and therefore highly unliked by c++11) and do not leak memory on this error. Remaining problem is unmapping such memory when firebird dynamic library (i.e. plugin) is unloaded - cleanup code does it's best to unmap linked blocks but if it fails (VMAs allocated by our library are mixed with allocated by others) memory leak appears unavoidable.

Solution was tested artificially reserving most of VMAs using code from https://stackoverflow.com/questions/43743555/munmap-failure-with-enomem-with-private-anonymous-mapping. After applying this fix SC server 2.5 stopped to fail randomly during massive deallocation of memory (was caused stopping TPC-C) and demonstrated stable behavior without memory leaks.

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Oct 31, 2017

Modified by: @AlexPeshkoff

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 2.5.8 [ 10809 ]

Fix Version: 3.0.3 [ 10810 ]

Fix Version: 4.0 Beta 1 [ 10750 ]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Oct 31, 2017

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Resolved [ 5 ]

QA Status: No test => Cannot be tested

Test Specifics: [Platform (Windows/Linux) specific]

@firebird-issue-importer
Copy link
Author

firebird-issue-importer commented Oct 31, 2017

Modified by: @pavel-zotov

status: Resolved [ 5 ] => Closed [ 6 ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment