New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MergeTree & SipHash checksum big-endian support #50276
MergeTree & SipHash checksum big-endian support #50276
Conversation
This is an automated comment for commit 14977e6 with description of existing statuses. It's updated for the latest CI running
|
0b788bb
to
1f49e25
Compare
This comment was marked as outdated.
This comment was marked as outdated.
72ebdfc
to
02c3e80
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
fc5f20f
to
5cfd462
Compare
5cfd462
to
1f5bdae
Compare
This comment was marked as outdated.
This comment was marked as outdated.
As there are currently many failing tests in ClickHouse's CI, a policy was put in place which prevents merging of PRs with red tests, even if unrelated. |
@rschu1ze thanks for the heads-up! |
11e37ca
to
8d836a6
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
8d836a6
to
f6850f0
Compare
This comment was marked as outdated.
This comment was marked as outdated.
035c3e6
to
c313b9e
Compare
7fe0179
to
1208e59
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
a1abfde
to
14977e6
Compare
The changes are meant to fix the
00961_checksums_in_system_parts_columns_table
functional test suite.Marks are now always serialized as little-endian to ensure interoperability between platforms. Composite structures are now transformed correctly from big-endian to little-endian when computing hashes using SipHash.
The handling of 128-bit checksums used for total checksum calculation consisting of two 64-bit unsigned integers was inconsistent in some cases: in big-endian mode CityHash returns them in a
lo
,hi
component order and only the values of the components are big-endian, whereas SipHash returns a more canonical version ashi
,lo
with both the order of components and their values following big-endian convention. However, when reusing the output data structure of CityHash (i.e.std::pair<uint64_t, uint64_t>
) SipHash now has aget128LoHi
function that follows thelo
,hi
component order no matter what endianness the platform is so that the resulting hashes are comparable without having to know which algorithm they were produced by.Changelog category (leave one):