Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SerializationString improve performance #57717

Conversation

kitaisreal
Copy link
Collaborator

@kitaisreal kitaisreal commented Dec 10, 2023

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Improve performance of string serialization.

@kitaisreal kitaisreal added the can be tested Allows running workflows for external contributors label Dec 10, 2023
@kitaisreal
Copy link
Collaborator Author

Prepare table:

CREATE TABLE test_table (id UInt64, value String) ENGINE=ReplacingMergeTree ORDER BY (id) SETTINGS index_granularity = 32768;
INSERT INTO test_table SELECT number, number % 15000 FROM numbers(1500000000);

Before:

SELECT max(length(value))
FROM test_table
FORMAT `Null`

0 rows in set. Elapsed: 0.855 sec. Processed 1.50 billion rows, 19.89 GB (1.75 billion rows/s., 23.27 GB/s.)
Peak memory usage: 1.24 MiB.

After:

SELECT max(length(value))
FROM test_table
FORMAT `Null`

0 rows in set. Elapsed: 0.691 sec. Processed 1.50 billion rows, 19.89 GB (2.17 billion rows/s., 28.79 GB/s.)
Peak memory usage: 1.17 MiB.

@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-performance Pull request with some performance improvements label Dec 10, 2023
@robot-ch-test-poll3
Copy link
Contributor

robot-ch-test-poll3 commented Dec 10, 2023

This is an automated comment for commit 6cab012 with description of existing statuses. It's updated for the latest CI running

⏳ Click here to open a full report in a separate page

Successful checks
Check nameDescriptionStatus
Mergeable CheckChecks if all other necessary checks are successful✅ success
Check nameDescriptionStatus
CI runningA meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR⏳ pending

@kitaisreal kitaisreal force-pushed the serialization-string-improve-performance branch from a0e1866 to 88c6ef6 Compare December 10, 2023 17:51
@alexey-milovidov
Copy link
Member

Please add the changelog entry and merge with master.

@alexey-milovidov alexey-milovidov self-assigned this Dec 10, 2023
@kitaisreal kitaisreal force-pushed the serialization-string-improve-performance branch from 88c6ef6 to 81250d1 Compare December 11, 2023 09:55
@kitaisreal kitaisreal added the force tests Force test ignoring fast test output. label Dec 11, 2023
Copy link
Member

@alexey-milovidov alexey-milovidov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a clear win!

@alexey-milovidov alexey-milovidov merged commit 3e52fcb into ClickHouse:master Dec 12, 2023
1 of 2 checks passed
@robot-clickhouse robot-clickhouse added the submodule changed At least one submodule changed in this PR. label Dec 12, 2023
yokofly added a commit to timeplus-io/proton that referenced this pull request Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
can be tested Allows running workflows for external contributors force tests Force test ignoring fast test output. pr-performance Pull request with some performance improvements submodule changed At least one submodule changed in this PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants