Use extent hooks for per query memory limit #1340
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces memory tracker with jemalloc extent hooks which fully works in case when jemalloc config is following:
MALLOC_CONF="retain:false,percpu_arena:percpu,oversize_threshold:1000000000000,muzzy_decay_ms:0,dirty_decay_ms:0" \
./configure \
--disable-cxx \
$COMMON_CONFIGURE_FLAGS \
--with-malloc-conf="retain:false,percpu_arena:percpu,oversize_threshold:1000000000000,muzzy_decay_ms:0,dirty_decay_ms:0"
This config will for jemalloc not to use lazy purge or MADV_FREE(muzzy_decay_ms=0 and dirty_decay_ms=0), it will force jemalloc not to use
custom arena for huge allocations (oversize_threshold) and it will force jemalloc not extend virtual memory indefinitely (retain=false)
and therefore call alloc hook when allocation actually takes place.
Only problem is if we do huge allocations which are not mapped on alloc directly, in that case jemalloc uses cache and we can overcounter
allocation size.
|
@Ignition @Josipmrden made one final fix on PR. It turns out there was a deadlocking issue. When one thread tried to register and map the thread id to the transaction id, an allocation request was made which ended up in getting tracker by thread id, which would result in deadlock since the lock was still locked, but ended up in a different function. To fix issue, I used utils::SkipList which is concurrent so no need for synchronization any more and everything now works. |
|
PR fixes this issue: #977 |
|
One more thing left to solve here:
|
Josipmrden
approved these changes
Oct 25, 2023
Ignition
approved these changes
Oct 25, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Each thread is mapped to one arena in jemalloc. Since we are working with a multi-thread environment where multiple threads can be mapped to the same arena, the thread ID is used as an identifier which is mapped to the transaction id. Each transaction can therefore have multiple threads that do allocation. Each thread is mapped internally to transaction id and has its own tracker. When a transaction with that id is done, the tracker is destroyed. If we have multiple queries inside an explicit transaction, each will have the same id, but they can't overwrite each other since queries are sequentially executed.
[master < Task] PR
To keep docs changelog up to date, one more thing to do:
What needs to be done