Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up countDigits #46187

Merged
merged 4 commits into from
Feb 10, 2023
Merged

Speed up countDigits #46187

merged 4 commits into from
Feb 10, 2023

Conversation

alexey-milovidov
Copy link
Member

@alexey-milovidov alexey-milovidov commented Feb 9, 2023

Changelog category (leave one):

  • Backward Incompatible Change

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Slightly improve performance of countDigits on realistic datasets. This closed #44518. In previous versions, countDigits(0) returned 0; now it returns 1, which is more correct, and follows the existing documentation.

Continuation of #44518.

@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-performance Pull request with some performance improvements label Feb 9, 2023
{
auto value = src_data[i].value;
if (unlikely(value < 0))
dst_data[i] = digits10<NativeT>(-static_cast<NativeT>(value));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI first casting to unsigned, then negating - is the only safe way to do it in C++.

@alexey-milovidov
Copy link
Member Author

Before:

milovidov@milovidov-desktop:~$ for column in {RegionID,ClientIP,WatchID,AdvEngineID}; do clickhouse-benchmark -i100 --query "SELECT sum(countDigits($column)) FROM hits"; done
Loaded 1 queries.

Queries executed: 100.

localhost:9000, queries 100, QPS: 36.253, RPS: 3625209320.385, MiB/s: 13829.076, result RPS: 36.253, result MiB/s: 0.000.

0.000%          0.024 sec.
10.000%         0.025 sec.
20.000%         0.025 sec.
30.000%         0.026 sec.
40.000%         0.026 sec.
50.000%         0.026 sec.
60.000%         0.027 sec.
70.000%         0.027 sec.
80.000%         0.028 sec.
90.000%         0.029 sec.
95.000%         0.033 sec.
99.000%         0.054 sec.
99.900%         0.090 sec.
99.990%         0.090 sec.

Queries executed: 100.

localhost:9000, queries 100, QPS: 34.905, RPS: 3490418577.064, MiB/s: 13314.890, result RPS: 34.905, result MiB/s: 0.000.

0.000%          0.026 sec.
10.000%         0.026 sec.
20.000%         0.027 sec.
30.000%         0.027 sec.
40.000%         0.027 sec.
50.000%         0.028 sec.
60.000%         0.028 sec.
70.000%         0.029 sec.
80.000%         0.030 sec.
90.000%         0.031 sec.
95.000%         0.032 sec.
99.000%         0.036 sec.
99.900%         0.066 sec.
99.990%         0.066 sec.

Queries executed: 100.

localhost:9000, queries 100, QPS: 20.091, RPS: 2009053467.947, MiB/s: 15327.862, result RPS: 20.091, result MiB/s: 0.000.

0.000%          0.047 sec.
10.000%         0.048 sec.
20.000%         0.048 sec.
30.000%         0.049 sec.
40.000%         0.049 sec.
50.000%         0.049 sec.
60.000%         0.049 sec.
70.000%         0.050 sec.
80.000%         0.051 sec.
90.000%         0.052 sec.
95.000%         0.053 sec.
99.000%         0.056 sec.
99.900%         0.073 sec.
99.990%         0.073 sec.


Queries executed: 100.

localhost:9000, queries 100, QPS: 58.320, RPS: 5831856159.607, MiB/s: 11123.383, result RPS: 58.320, result MiB/s: 0.000.

0.000%          0.014 sec.
10.000%         0.015 sec.
20.000%         0.015 sec.
30.000%         0.016 sec.
40.000%         0.017 sec.
50.000%         0.017 sec.
60.000%         0.017 sec.
70.000%         0.017 sec.
80.000%         0.018 sec.
90.000%         0.018 sec.
95.000%         0.019 sec.
99.000%         0.036 sec.
99.900%         0.044 sec.
99.990%         0.044 sec.

After:

milovidov@milovidov-desktop:~$ for column in {RegionID,ClientIP,WatchID,AdvEngineID}; do clickhouse-benchmark -i100 --query "SELECT sum(countDigits($column)) FROM hits"; done
Loaded 1 queries.

Queries executed: 100.

localhost:9000, queries 100, QPS: 42.361, RPS: 4235954485.892, MiB/s: 16158.884, result RPS: 42.361, result MiB/s: 0.000.

0.000%          0.019 sec.
10.000%         0.021 sec.
20.000%         0.021 sec.
30.000%         0.022 sec.
40.000%         0.022 sec.
50.000%         0.022 sec.
60.000%         0.023 sec.
70.000%         0.023 sec.
80.000%         0.024 sec.
90.000%         0.027 sec.
95.000%         0.030 sec.
99.000%         0.051 sec.
99.900%         0.053 sec.
99.990%         0.053 sec.

Queries executed: 100.

localhost:9000, queries 100, QPS: 42.068, RPS: 4206646165.442, MiB/s: 16047.082, result RPS: 42.068, result MiB/s: 0.000.

0.000%          0.020 sec.
10.000%         0.021 sec.
20.000%         0.021 sec.
30.000%         0.022 sec.
40.000%         0.022 sec.
50.000%         0.023 sec.
60.000%         0.023 sec.
70.000%         0.024 sec.
80.000%         0.025 sec.
90.000%         0.027 sec.
95.000%         0.029 sec.
99.000%         0.044 sec.
99.900%         0.048 sec.
99.990%         0.048 sec.

Queries executed: 100.

localhost:9000, queries 100, QPS: 19.724, RPS: 1972324306.630, MiB/s: 15047.640, result RPS: 19.724, result MiB/s: 0.000.

0.000%          0.048 sec.
10.000%         0.049 sec.
20.000%         0.049 sec.
30.000%         0.050 sec.
40.000%         0.050 sec.
50.000%         0.050 sec.
60.000%         0.051 sec.
70.000%         0.051 sec.
80.000%         0.052 sec.
90.000%         0.053 sec.
95.000%         0.054 sec.
99.000%         0.057 sec.
99.900%         0.065 sec.
99.990%         0.065 sec.

Queries executed: 100.

localhost:9000, queries 100, QPS: 65.619, RPS: 6561708923.538, MiB/s: 12515.467, result RPS: 65.619, result MiB/s: 0.001.

0.000%          0.012 sec.
10.000%         0.012 sec.
20.000%         0.014 sec.
30.000%         0.015 sec.
40.000%         0.015 sec.
50.000%         0.015 sec.
60.000%         0.015 sec.
70.000%         0.015 sec.
80.000%         0.016 sec.
90.000%         0.017 sec.
95.000%         0.020 sec.
99.000%         0.023 sec.
99.900%         0.036 sec.
99.990%         0.036 sec.

@robot-ch-test-poll2 robot-ch-test-poll2 added pr-backward-incompatible Pull request with backwards incompatible changes and removed pr-performance Pull request with some performance improvements labels Feb 9, 2023
@nickitat nickitat self-assigned this Feb 9, 2023
@alexey-milovidov alexey-milovidov merged commit 786aa06 into master Feb 10, 2023
@alexey-milovidov alexey-milovidov deleted the speed-up-count-digits branch February 10, 2023 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-backward-incompatible Pull request with backwards incompatible changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants