Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Data-Correctness Bug in GTE Comparison in BinaryOperatorTransformFunction #9461

Merged
merged 3 commits into from Sep 27, 2022

Conversation

ankitsultana
Copy link
Contributor

@ankitsultana ankitsultana commented Sep 24, 2022

fixes #9460

The BinaryOperatorTransformFunction::compare methods already call getIntResult, but the places where it's called calls getIntResult again.

https://github.com/apache/pinot/blob/master/pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/BinaryOperatorTransformFunction.java#L371

Say the comparison is: 10 >= 20

Then, the call chain is (bottom-up): Double.compare(10, 20) ==> -1 ==> getIntResult(-1) ==> 0 ==> getIntResult(0) ==> 1

Which means 10 >= 20 gets evaluated to true

cc: @yupeng9 @chenboat

Copy link
Contributor

@Jackie-Jiang Jackie-Jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! You may add a test case in BinaryOperatorTransformFunctionTest

Copy link
Contributor

@jackjlli jackjlli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. +1 on adding a unit test for this.

@ankitsultana
Copy link
Contributor Author

Added a UT. Existing tests weren't able to catch this case since this only happens when the argument types are:

  1. float, long
  2. double, long
  3. long, float
  4. long, double

I also found a bug in my PR. I had only updated calls for one of the compare methods. Caught that bug via the UT I added 😓

Copy link
Contributor

@yupeng9 yupeng9 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

@codecov-commenter
Copy link

codecov-commenter commented Sep 27, 2022

Codecov Report

Merging #9461 (2bea20c) into master (96f4d1e) will decrease coverage by 0.04%.
The diff coverage is 66.66%.

@@             Coverage Diff              @@
##             master    #9461      +/-   ##
============================================
- Coverage     69.88%   69.83%   -0.05%     
- Complexity     5127     5134       +7     
============================================
  Files          1909     1910       +1     
  Lines        101749   101769      +20     
  Branches      15440    15444       +4     
============================================
- Hits          71104    71071      -33     
- Misses        25627    25677      +50     
- Partials       5018     5021       +3     
Flag Coverage Δ
integration1 25.96% <0.00%> (-0.07%) ⬇️
integration2 24.67% <0.00%> (-0.07%) ⬇️
unittests1 67.13% <66.66%> (+0.01%) ⬆️
unittests2 15.51% <0.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...form/function/BinaryOperatorTransformFunction.java 45.36% <66.66%> (+5.49%) ⬆️
...g/apache/pinot/server/api/resources/ErrorInfo.java 0.00% <0.00%> (-100.00%) ⬇️
...data/manager/realtime/DefaultSegmentCommitter.java 0.00% <0.00%> (-80.00%) ⬇️
...t/server/api/resources/DefaultExceptionMapper.java 0.00% <0.00%> (-75.00%) ⬇️
...er/api/resources/LLCSegmentCompletionHandlers.java 43.56% <0.00%> (-18.82%) ⬇️
...data/manager/realtime/SegmentCommitterFactory.java 88.23% <0.00%> (-11.77%) ⬇️
.../pinot/server/api/resources/TableSizeResource.java 80.00% <0.00%> (-8.00%) ⬇️
...transform/function/IsNotNullTransformFunction.java 67.85% <0.00%> (-7.15%) ⬇️
...altime/ServerSegmentCompletionProtocolHandler.java 51.88% <0.00%> (-6.61%) ⬇️
.../predicate/NotEqualsPredicateEvaluatorFactory.java 74.03% <0.00%> (-5.77%) ⬇️
... and 35 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@yupeng9 yupeng9 merged commit 83b7f15 into apache:master Sep 27, 2022
61yao pushed a commit to 61yao/pinot that referenced this pull request Oct 3, 2022
…Function (apache#9461)

* Fix Bug in Handling GTE Comparison in BinaryOperatorTransformFunction

* Add UT

* Fix bug and add another test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Using GTE Comparison in Projection Leads to Incorrect Results
5 participants