Please sign in to comment.
[SPARK-24879][SQL] Fix NPE in Hive partition pruning filter pushdown
## What changes were proposed in this pull request? We get a NPE when we have a filter on a partition column of the form `col in (x, null)`. This is due to the filter converter in HiveShim not handling `null`s correctly. This patch fixes this bug while still pushing down as much of the partition pruning predicates as possible, by filtering out `null`s from any `in` predicate. Since Hive only supports very simple partition pruning filters, this change should preserve correctness. ## How was this patch tested? Unit tests, manual tests Author: William Sheu <firstname.lastname@example.org> Closes #21832 from PenguinToast/partition-pruning-npe. (cherry picked from commit bbd6f0c) Signed-off-by: Xiao Li <email@example.com>
- Loading branch information...
Showing with 32 additions and 1 deletion.