[SPARK-12218] [SQL] [Backport-1.5] Fixed the Parquet's filter generation rule when Not is included in Parquet filter pushdown

[gatorsmile]

Added the test case that can cause data loss in the following scenario:

When applying the operator Not, the current generation rule for Parquet filters simply applies Not to all the inclusive/underlying filters.

Note: will submit the fix after the test case failure.

[SparkQA]

Test build #47891 has finished for PR 10344 at commit 660eef5.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

checkAnswer(
    sqlContext.read.parquet(path).where("not (a = 2 and b in ('1'))"),
    (1 to 5).map(i => Row(i, (i%2).toString)))

The above test case failed.

[info]   == Results ==
[info]   !== Correct Answer - 5 ==   == Spark Answer - 4 ==
[info]    [1,1]                      [1,1]
[info]   ![2,0]                      [3,1]
[info]   ![3,1]                      [4,0]
[info]   ![4,0]                      [5,1]
[info]   ![5,1]

@yhuai Could you take a look at it? Should I merge my fix? Thanks!

[yhuai]

I see. Thanks! Let's merge your fix.

[gatorsmile]

This is just an one-line fix. It does not include the pushdown for IN. @yhuai If you want it, I also can merge that part and the related test cases. Thanks!

[SparkQA]

Test build #47899 has finished for PR 10344 at commit 7d298fe.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #47941 has finished for PR 10344 at commit 65c5ad7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.