Skip to content

Support reading var-length type (STRING/BYTES/BIG_DECIMAL) as murmur3 32/64/128 bits hash#15213

Merged
Jackie-Jiang merged 1 commit intoapache:masterfrom
Jackie-Jiang:read_murmur3
Mar 30, 2025
Merged

Support reading var-length type (STRING/BYTES/BIG_DECIMAL) as murmur3 32/64/128 bits hash#15213
Jackie-Jiang merged 1 commit intoapache:masterfrom
Jackie-Jiang:read_murmur3

Conversation

@Jackie-Jiang
Copy link
Contributor

This is useful for calculating distinct count

@codecov-commenter
Copy link

codecov-commenter commented Mar 6, 2025

Codecov Report

Attention: Patch coverage is 11.58537% with 145 lines in your changes missing coverage. Please review.

Project coverage is 63.15%. Comparing base (59551e4) to head (fce300b).
Report is 1944 commits behind head on master.

Files with missing lines Patch % Lines
...java/org/apache/pinot/core/common/DataFetcher.java 0.00% 36 Missing ⚠️
...core/operator/docvalsets/TransformBlockValSet.java 0.00% 18 Missing ⚠️
...al/segment/index/readers/BigDecimalDictionary.java 0.00% 12 Missing ⚠️
...t/local/segment/index/readers/BytesDictionary.java 0.00% 12 Missing ⚠️
.../local/segment/index/readers/StringDictionary.java 0.00% 12 Missing ⚠️
...che/pinot/segment/spi/index/reader/Dictionary.java 0.00% 12 Missing ⚠️
...ache/pinot/spi/utils/hash/MurmurHashFunctions.java 54.54% 8 Missing and 2 partials ⚠️
...a/org/apache/pinot/core/common/DataBlockCache.java 0.00% 9 Missing ⚠️
...ore/operator/docvalsets/ProjectionBlockValSet.java 0.00% 9 Missing ⚠️
...pache/pinot/segment/local/io/util/ValueReader.java 50.00% 6 Missing ⚠️
... and 3 more
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #15213      +/-   ##
============================================
+ Coverage     61.75%   63.15%   +1.40%     
- Complexity      207     1375    +1168     
============================================
  Files          2436     2814     +378     
  Lines        133233   159164   +25931     
  Branches      20636    24376    +3740     
============================================
+ Hits          82274   100524   +18250     
- Misses        44911    51056    +6145     
- Partials       6048     7584    +1536     
Flag Coverage Δ
custom-integration1 100.00% <ø> (+99.99%) ⬆️
integration 100.00% <ø> (+99.99%) ⬆️
integration1 100.00% <ø> (+99.99%) ⬆️
integration2 0.00% <ø> (ø)
java-11 63.10% <11.58%> (+1.39%) ⬆️
java-21 63.13% <11.58%> (+1.50%) ⬆️
skip-bytebuffers-false 63.15% <11.58%> (+1.40%) ⬆️
skip-bytebuffers-true 63.08% <11.58%> (+35.35%) ⬆️
temurin 63.15% <11.58%> (+1.40%) ⬆️
unittests 63.15% <11.58%> (+1.40%) ⬆️
unittests1 56.18% <11.58%> (+9.28%) ⬆️
unittests2 33.80% <4.26%> (+6.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jackie-Jiang Jackie-Jiang force-pushed the read_murmur3 branch 2 times, most recently from 4247303 to 5a7c5a2 Compare March 6, 2025 23:41
@Jackie-Jiang Jackie-Jiang changed the title Support reading var-length type (STRING/BYTES/BIG_DECIMAL) as murmur3 128 bits hash Support reading var-length type (STRING/BYTES/BIG_DECIMAL) as murmur3 32/64/128 bits hash Mar 28, 2025
@Jackie-Jiang Jackie-Jiang force-pushed the read_murmur3 branch 2 times, most recently from b27a159 to 93664b9 Compare March 28, 2025 01:58
@Jackie-Jiang Jackie-Jiang merged commit 7c9f3ae into apache:master Mar 30, 2025
22 checks passed
@Jackie-Jiang Jackie-Jiang deleted the read_murmur3 branch March 30, 2025 00:21
@xiangfu0 xiangfu0 linked an issue Oct 28, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add MurmurHash SQL functions

3 participants