[SPARK-16344][SQL][BRANCH-1.6] Decoding Parquet array of struct with a single field named "element"

[liancheng]

What changes were proposed in this pull request?

Please refer to SPARK-16344 for details about this issue.

How was this patch tested?

New test case added in ParquetQuerySuite.

[liancheng]

This case branch is essential for the bug fix. Basically, it matches the standard 3-level layout first before trying to match the legacy 2-level layout, so that the "element" syntactic group in Parquet LIST won't be mistaken for the "element" field in the nested struct.

[liancheng]

cc @yhuai

[SparkQA]

Test build #61611 has finished for PR 14013 at commit 9620b48.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

@rdblue Would you mind to help review this one? My initial investigation suggested that parquet-avro probably suffers the same issue. Will file a parquet-mr JIRA ticket soon if that's true.

[SparkQA]

Test build #61612 has finished for PR 14013 at commit c40bccb.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

@rdblue Verified that parquet-avro also suffers from this issue. Filed PARQUET-651 to track it.

[SparkQA]

Test build #61812 has finished for PR 14013 at commit e44451e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #61838 has finished for PR 14013 at commit 70b2e9c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #62052 has finished for PR 14013 at commit b942dca.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

I'm closing this one since we decided to fix this in master only.