-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix partition ID byte order for s390x #47769
Conversation
@@ -261,8 +261,11 @@ String MergeTreePartition::getID(const Block & partition_key_sample) const | |||
hash.get128(hash_data); | |||
result.resize(32); | |||
for (size_t i = 0; i < 16; ++i) | |||
#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if constexpr (std::endian::native == std::endian::big)
would be nicer
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if constexpr (std::endian::native == std::endian::big)
only works when it is in a template function. Since getID() is not a template function, using this way would cause compiling error.
Also let's change the changelog category to "Backward Incompatible Change" and make the changelog entry more detailed |
And do we have the same issue with other usages of ClickHouse/src/Parsers/Access/ParserCreateUserQuery.cpp Lines 192 to 201 in 9920a52
|
This issue is unique because getID() uses SipHash128 to generate 128 bit wide integers for the partition ID. Even though the generated 128 bit integers are the same for both little-endian and big-endian machines, directly using the data of 128 bit integers would cause byte order issue in s390x. |
eb7da02
to
1462606
Compare
Done. |
1462606
to
ed1ff02
Compare
This is not a backward incompatible change. |
@alexey-milovidov, why? It changes partition ids on s390x platform. Of course, it does not affect other platforms, but it is backward incompatible for s390x. |
1 similar comment
@alexey-milovidov, why? It changes partition ids on s390x platform. Of course, it does not affect other platforms, but it is backward incompatible for s390x. |
s390x is not supported for ClickHouse. |
On s390x, the partition ID has reversed byte order compared to those on little-endian machines. The fix is to reverse the byte order so that partition IDs keep the same on both little-endian and big-endian machines.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fixed the byte order of partition IDs on s390x so that partition IDs keep the same on both little-endian and big-endian machines. This change does not affect other platforms.
Documentation entry for user-facing changes