Commit c456cfe
[SPARK-21529][4.X][SQL] Improve the error message for unsupported Hive union type
### What changes were proposed in this pull request?
Backport of #56775 to `branch-4.x` because the original change conflicts on this branch.
Detect unsupported Hive `uniontype<...>` values when converting Hive `FieldSchema` types to Spark SQL types and raise a dedicated `UNSUPPORTED_HIVE_TYPE` error instead of the generic `CANNOT_RECOGNIZE_HIVE_TYPE` parser error.
This is a cherry-pick of the merged master commit c90cad6. The only conflict was in `error-conditions.json`: `branch-4.x` does not have the `UNSUPPORTED_HIVE_FUNCTION_TYPE` / `UNSUPPORTED_HIVE_METASTORE_VERSION_FOR_JAVA` entries that exist on master, so the new `UNSUPPORTED_HIVE_TYPE` entry is placed directly between `UNSUPPORTED_GROUPING_EXPRESSION` and `UNSUPPORTED_INSERT`. The Scala changes apply unchanged.
### Why are the changes needed?
Spark SQL does not support Hive union types. Today the failure message comes from the parser path and does not clearly identify that the Hive union type is unsupported.
### Does this PR introduce _any_ user-facing change?
Yes. Reading a Hive table column that uses `uniontype<...>` now reports `UNSUPPORTED_HIVE_TYPE` with the offending Hive type and column name.
### How was this patch tested?
Cherry-picked from the merged master commit c90cad6, which passed CI and review as #56775. The production Scala hunks apply unchanged on `branch-4.x`; only the `error-conditions.json` entry placement differed and was re-validated (valid JSON, alphabetical ordering, one structural token per line). CI here runs `HiveClientImplSuite` and the `SparkThrowableSuite` "Error conditions are correctly formatted" golden check.
### Was this patch authored or co-authored using generative AI tooling?
Yes. GitHub Copilot assisted with preparing and validating this change.
Closes #56929 from AgenticSpark/agenticspark/SPARK-21529-branch-4.x.
Authored-by: AgenticSpark <jianglie2023@gmail.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>1 parent 5f39cc2 commit c456cfe
4 files changed
Lines changed: 67 additions & 0 deletions
File tree
- common/utils/src/main/resources/error
- sql
- catalyst/src/main/scala/org/apache/spark/sql/errors
- hive/src
- main/scala/org/apache/spark/sql/hive/client
- test/scala/org/apache/spark/sql/hive/client
Lines changed: 6 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8641 | 8641 | | |
8642 | 8642 | | |
8643 | 8643 | | |
| 8644 | + | |
| 8645 | + | |
| 8646 | + | |
| 8647 | + | |
| 8648 | + | |
| 8649 | + | |
8644 | 8650 | | |
8645 | 8651 | | |
8646 | 8652 | | |
| |||
Lines changed: 7 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1696 | 1696 | | |
1697 | 1697 | | |
1698 | 1698 | | |
| 1699 | + | |
| 1700 | + | |
| 1701 | + | |
| 1702 | + | |
| 1703 | + | |
| 1704 | + | |
| 1705 | + | |
1699 | 1706 | | |
1700 | 1707 | | |
1701 | 1708 | | |
| |||
Lines changed: 5 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1139 | 1139 | | |
1140 | 1140 | | |
1141 | 1141 | | |
| 1142 | + | |
| 1143 | + | |
| 1144 | + | |
| 1145 | + | |
| 1146 | + | |
1142 | 1147 | | |
1143 | 1148 | | |
1144 | 1149 | | |
| |||
Lines changed: 49 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
0 commit comments