oatpp::base::memory::MemoryPool::freeByEntryHeader()]: Invalid EntryHeader #308
Comments
|
Hello @pcapdump , The memory pool error You may disable pool allocations with The number of controllers doesn't really matter, you may have as many controllers as you like.
Regards, |
|
Thanks for answer, Leonid.
Aren't you mussing up memory leak and memory corruption? Memory leak can crash application only when there is out-of-memory condition (when application consumes several Gigabytes of RAM), which is not our case.
simple
no, it's closed source. |
|
Hey,
In your case, it seems that the application was still using a resource that was already deleted. Also, memory pool is static, so its kinda weird |
Well, this could be the root of the issue. https://github.com/oatpp/oatpp/blob/master/src/oatpp/core/data/stream/ChunkedBuffer.hpp#L55 Destruction order of static (=global) objects on application exit is undefined. BTW, your code violates Google coding conventions for C++: static complex objects should be allocated with Above is the proper Singleton pattern for your code as I transformed static object with undefined destruction time to static pointer which is never freed. |
|
Hey, Thanks for the example code-snippet and for recommendations.
This doesn't look like the root of an issue to me. From the pools' point of view, it's a memory leak. |
|
About the above snippet:
This sounds reasonable. static oatpp::base::memory::ThreadDistributedMemoryPool& getSegemntPool(){
static auto pool = new oatpp::base::memory::ThreadDistributedMemoryPool(CHUNK_POOL_NAME, CHUNK_ENTRY_SIZE, CHUNK_CHUNK_SIZE);
return *pool;
}Also, please feel free to send PRs with changes you believe will improve oatpp. Regards, |
|
Hey @pcapdump , Thanks for the update, and please let me know in case you find something else. Disabling memory pools is not a big deal and won't affect performance. Pools were added in the very first release of oatpp, and since then a lot has changed. I plan to remove pools in future releases and reintroduce them after careful testing where it's needed. However, I still suspect there must be some resource leaking in your code. Can you please share your code where you create components, and run the server and where you stop your server? |
|
Hey, do you have the possibility to run your application with Valgrind? And can you create a minimal working example of the error? |
|
Hi! I reproduced the issue with Here's the result (I omitted some private info): I interpret it as follows:
Then I was a bit frustrated why do I get a static MemoryPool object cleanup if I changed it to be a static pointer which should be never deleted (see my patch above). Then I changed the code in all the places and the issue was fixed. Please see patch attached. Sorry, it's not a final patch as it contains a mixture of different ideas and workarounds which need to be cleaned. |
|
Hey @pcapdump , Thanks for investigating this issue and for the patch!
It's possible to do with forks:
You can do everything on Github UI with just button-clicking.
I've reviewed the Patch, and it looks good to me. static oatpp::base::memory::MemoryPool& getPool(const AllocatorPoolInfo& info){
#ifndef OATPP_COMPAT_BUILD_NO_THREAD_LOCAL
/* memory leak because of thread_local */
static thread_local oatpp::base::memory::MemoryPool* pool = new oatpp::base::memory::MemoryPool(info.poolName, sizeof(T), info.poolChunkSize);
#else
static auto pool = new oatpp::base::memory::MemoryPool(info.poolName, sizeof(T), info.poolChunkSize);
#endif
return *pool;
}
Oat++ min C++ version is 11, and starting from C++11 such singleton is guaranteed to be thread safe static MemoryPool& getPool() {
static auto pool = new MemoryPool();
return *pool;
}So I'm not sure if we should introduce additional safety constructions. Well, again, unless there is a known case where it doesn't work as expected by the standard. Please feel free to send the PR. Best Regards, |
|
Leonid, please review PR #314 Alas, it doesn't work without With |
|
Still having issue with PR #314 : Still |
|
@pcapdump , hey I'll be able to take a look closer in 3 hours. |
It looks like you're right. However I don't know how to express Do you think you could fix it somehow at next release? Please advice what's the best way to proceed. |
Issue #308: oatpp::base::memory::MemoryPool::freeByEntryHeader()]: Invalid EntryHeader
|
Hey @pcapdump , I like the changes in the PR so it's merged now.
Ok, now I got what was the original issue :) You are stopping the application with an interrupt signal thus it doesn't go through the proper cleanup process and memory pools report a leak. With a signal handler, you can gracefully stop the server like here and no error should be reported. $ ./my-project-exe
I |2020-10-02 02:55:04 1601596504860169| MyApp:Server running on port 8000
^C I |2020-10-02 02:55:06 1601596506422256| MyApp:Sig received!
$ _
From my side, I would recommend having your own fork and a build pipeline above it. Thus you'll be able:
This approach will give the best synergy. Yes, the pools will be addressed in the next releases. We have some code I want to clean up. Pools also should be tunned a bit. Regards, |
|
Hey @pcapdump , I'm curious if you could describe your oatpp use-case - it helps to better understand our users. Also, please feel free to join the devs chat on Gitter - https://gitter.im/oatpp-framework/Lobby Regards |
|
Thanks for the answer, @lganzzzo
Sorry, but I thought that you understood that the issue here is not a leak in our application, but a
This solution doesn't fit us well, because:
So the outcome of this conversation is that you don't plan to fix it? |
|
Hey @pcapdump ,
From oatpp perspective, it's exactly a memory leak. And Asan's output shows that you are using oatpp incorrectly. How resource life-cycle should look like in a typical oatpp applicationApp start
App exit
Whatever reason that leads to the Router being destroyed in uncontrolled order during static resource deallocation -
All user-created oatpp objects including the
The output of The correct way of doing custom oatpp based library
IMHO libraries should provide tools and resources for the user application. The corporate world
In the corporate world, this is totally understandable and considered to be "normal". However, what is strange in the corporate world - is when The Solution
Little hack - little blood.
Yes. |
|
I'm closing this. |
Hi!
I observe this exception when I'm stopping the application with OATPP:
I was able to get some more logs of it:
Backtrace:
The issue started to occur after I added 2nd controller.
There was no such issue with single controller.
OATPP version is 1.1.0
After I disabled pool allocation the issue doesn't reproduce anymore:
Looks to be connected with #173
Is this a bug at OATPP custom pool memory allocator?
Thanks in advance.
The text was updated successfully, but these errors were encountered: