[SPARK-58672][SQL] Reject CHAR/VARCHAR map keys in variant conversions - #57878
Open
ueshin wants to merge 2 commits into
Open
[SPARK-58672][SQL] Reject CHAR/VARCHAR map keys in variant conversions#57878ueshin wants to merge 2 commits into
ueshin wants to merge 2 commits into
Conversation
Member
Author
uros-b
approved these changes
Aug 9, 2026
HyukjinKwon
approved these changes
Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Update
VariantGet.checkDataTypeto reject map types whose keys areCHARorVARCHAR. This validation is shared byvariant_getandto_variant_object.Add tests covering
STRING,CHAR, andVARCHARas top-level target types and as array elements, map keys/values, and struct fields. The tests cover bothvariant_gettarget types andto_variant_objectinput types where applicable.Why are the changes needed?
CHARandVARCHARextendStringType. As a result, the existingMapType(_: StringType, ...)pattern incorrectly accepts them as map key types, even thoughVariantGet.checkDataTypeexplicitly rejectsCHARandVARCHARin other positions. Since this validation is also used byto_variant_object, the same issue affects both variant conversion paths.Does this PR introduce any user-facing change?
Yes.
variant_getandto_variant_objectnow reject maps withCHARorVARCHARkey types during analysis. Maps withSTRINGkeys remain supported.CHARandVARCHARin array elements, map values, and struct fields were already rejected; the tests now cover these cases explicitly.How was this patch tested?
Added unit tests covering positive
STRINGcases and negativeCHAR/VARCHARcases forvariant_getandto_variant_object, including array elements, map keys/values, and struct fields.Was this patch authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex (GPT-5)