New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Aggregation with structural type causes GC pressure and excessive cross region reference #9553
Comments
For This idea should also be applicable to the following aggregation functions: |
Similarly, for While |
Per discussion with @highker and @haozhun . There is an easier solution for The issue is currently we use Lines 201 to 207 in 9a42ff1
Instead of slice the large block into In a long term, we might want to be able to perform automatic compaction. |
Update: We have been able to reproduce the issue by a query contains a lot of
@nezihyigitbasi and @highker helped to find mitigations by tuning |
This issue has been automatically marked as stale because it has not had any activity in the last 2 years. If you feel that this issue is important, just comment and the stale tag will be removed; otherwise it will be closed in 7 days. This is an attempt to ensure that our open issues remain valuable and relevant so that we can keep track of what needs to be done and prioritize the right things. |
This issue has been automatically marked as stale because it has not had any activity in the last 2 years. If you feel that this issue is important, just comment and the stale tag will be removed; otherwise it will be closed in 7 days. This is an attempt to ensure that our open issues remain valuable and relevant so that we can keep track of what needs to be done and prioritize the right things. |
We have seen the following two issues in production when performing aggregations with structural type with huge number of groups. There are two separate cases:
We see a lot of
SingleMapBlock
when the aggregation function is likeMAX_BY(map, x)
orARBITRARY(map)
, since anSingleMapBlock
will be used for each group state.This issue can also happen for
SingleRowBlock
(for row type) orXArrayBlock
(for array type). We have also seen a massive number ofSlice
in heap forMAX(varchar)
.We see a lot of
MapBlockBuilder
forARRAY_AGG(map)
.In general, this doesn't restricts to map type, since we could see a lot of
BlockBuilder
forARRAY_AGG(x)
on any type, as long as the aggregation produces many groups.The text was updated successfully, but these errors were encountered: