Alex asked me to open special ticket for this issue.
Host has several databases which are served by single FB instance. Total number of attachments in work time is ~550.
After some time of normal work following message "Operating system call munmap failed. Error code 12" - did appear in firebird.log four times, and after that fb_smp_server was crashed.
I've sent stack-trace and additional info by e-mail, 18.10.2017 23:48.
summary: Message "Operating system call munmap failed. Error code 12" can apper in firebird.log under heavy load (2.5.x, CS, SC) => Message "Operating system call munmap failed. Error code 12" can appear in firebird.log under heavy load (2.5.x, CS, SC)
Certainly one can't change Linux behavior regarding splitting VMA (Virtual Memory Area) structures in munmap() call.
What we get with this fix - memory block which can't be unmapped due to ENOMEM error is saved in linked list in MemoryPool and reused as soon as possible. I.e. we avoid throwing exception (which often happens in dtors and therefore highly unliked by c++11) and do not leak memory on this error. Remaining problem is unmapping such memory when firebird dynamic library (i.e. plugin) is unloaded - cleanup code does it's best to unmap linked blocks but if it fails (VMAs allocated by our library are mixed with allocated by others) memory leak appears unavoidable.