Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add server setting aggregate_function_group_array_max_element_size. #53550

Merged
merged 4 commits into from Aug 18, 2023

Conversation

KochetovNicolai
Copy link
Member

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Added server setting aggregate_function_group_array_max_element_size. This setting is used to limit array size for groupArray function at serialization. The default value is 16777215.

Note: initially, the value was hardcoded. The check was working on deserialization only. It was possible to insert some data, but never read in again. The server setting is a work-around for such a case.

Repro:

create table tab3 (x AggregateFunction(groupArray, Array(UInt8))) engine = MergeTree order by tuple();
insert into tab3 select groupArrayState([zero]) from zeros(0x1000000);
select length(groupArrayMerge(x)) from tab3;

SELECT length(groupArrayMerge(x))
FROM tab3

Query id: b98ae84b-deaf-4c31-834c-741166411da7


0 rows in set. Elapsed: 0.031 sec. 

Received exception from server (version 23.8.1):
Code: 128. DB::Exception: Received from localhost:9000. DB::Exception: Too large array size (maximum: 16777215): (while reading from part /home/ubuntu/test/clickhouse/store/5e9/5e93da3c-7033-47dd-8011-7ecb427f50fa/all_1_1_0/ in table default.tab3 (5e93da3c-7033-47dd-8011-7ecb427f50fa) located on disk default of type local, from mark 0 with max_rows_to_read = 1): While executing MergeTreeInOrder. (TOO_LARGE_ARRAY_SIZE)

@robot-ch-test-poll1 robot-ch-test-poll1 added the pr-improvement Pull request with some product improvements label Aug 18, 2023
@robot-ch-test-poll1
Copy link
Contributor

robot-ch-test-poll1 commented Aug 18, 2023

This is an automated comment for commit 26c7dd2 with description of existing statuses. It's updated for the latest CI running
The full report is available here
The overall status of the commit is 🟡 pending

Check nameDescriptionStatus
CI runningA meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR🟡 pending
Mergeable CheckChecks if all other necessary checks are successful🟢 success

@nikitamikhaylov nikitamikhaylov changed the title Add server setting aggregate_function_group_array_max_element_size. Add server setting aggregate_function_group_array_max_element_size. Aug 18, 2023
@nikitamikhaylov nikitamikhaylov self-assigned this Aug 18, 2023
@KochetovNicolai KochetovNicolai merged commit c99c311 into master Aug 18, 2023
4 of 5 checks passed
@KochetovNicolai KochetovNicolai deleted the group_array_max_element_size branch August 18, 2023 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants