-
Notifications
You must be signed in to change notification settings - Fork 477
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ORC-1146: Float category missing check if the statistic sum is a finite value #1077
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM. Thank you, @guiyanakuang .
cc @williamhyun and @pgaref
…te value ### What changes were proposed in this pull request? This pr is aimed at checking whether the float category statistic sum has a finite value. ### Why are the changes needed? When the orc float category is written with NaN, pushing down is not supported. ### How was this patch tested? Added unit test. Closes #1077 Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 93a3505) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
I deleted my previous comment about the branch-1.7 test result. At a clean build, I verified this patch in branch-1.7 and backported. Thank you again, @guiyanakuang . Could you re-run your integration tests, @williamhyun ? |
BTW, @guiyanakuang . Could you check branch-1.6 too when you have a time? We backported the following in branch-1.6. I guess we might have the same issue there. |
Thanks @dongjoon-hyun . I checked the branch-1.6 and confirmed it has the same problem. This pr can just cherry pick and fix it without any conflicts. |
Thank you, @guiyanakuang . |
…te value ### What changes were proposed in this pull request? This pr is aimed at checking whether the float category statistic sum has a finite value. ### Why are the changes needed? When the orc float category is written with NaN, pushing down is not supported. ### How was this patch tested? Added unit test. Closes apache#1077 Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
What changes were proposed in this pull request?
This pr is aimed at checking whether the float category statistic sum has a finite value.
Why are the changes needed?
When the orc float category is written with NaN, pushing down is not supported.
How was this patch tested?
Added unit test.