Avoid MEMORY_LIMIT_EXCEEDED during INSERT into Buffer with AggregateFunction #35072
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
robot-clickhouse
added
the
pr-improvement
azat
force-pushed
the
buffer-memory-tracker-leak
branch
from
8be9022
to
106111d
Compare
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
…unction In case of Buffer table has columns of AggregateFunction type, aggregate states for such columns will be allocated from the query context but those states can be destroyed from the server context (in case of background flush), and thus memory will be leaked from the query since aggregate states can be shared, and eventually this will lead to MEMORY_LIMIT_EXCEEDED error. To avoid this, prohibit sharing the aggregate states. But note, that this problem only about memory accounting, not memory usage itself. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
azat
force-pushed
the
buffer-memory-tracker-leak
branch
from
106111d
to
e2960e1
Compare
alexey-milovidov
approved these changes
Mar 13, 2022
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
antonio2368
approved these changes
Mar 14, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Avoid possible
MEMORY_LIMIT_EXCEEDEDduringINSERTintoBufferwithAggregateFunctionIn case of Buffer table has columns of AggregateFunction type,
aggregate states for such columns will be allocated from the query
context but those states can be destroyed from the server context (in
case of background flush), and thus memory will be leaked from the query
since aggregate states can be shared, and eventually this will lead to
MEMORY_LIMIT_EXCEEDED error.
To avoid this, prohibit sharing the aggregate states.
But note, that this problem only about memory accounting, not memory
usage itself.