Skip to content

[SPARK-58672][SQL] Reject CHAR/VARCHAR map keys in variant conversions - #57878

Open
ueshin wants to merge 2 commits into
apache:masterfrom
ueshin:variant
Open

[SPARK-58672][SQL] Reject CHAR/VARCHAR map keys in variant conversions#57878
ueshin wants to merge 2 commits into
apache:masterfrom
ueshin:variant

Conversation

@ueshin

@ueshin ueshin commented Aug 9, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Update VariantGet.checkDataType to reject map types whose keys are CHAR or VARCHAR. This validation is shared by variant_get and to_variant_object.

Add tests covering STRING, CHAR, and VARCHAR as top-level target types and as array elements, map keys/values, and struct fields. The tests cover both variant_get target types and to_variant_object input types where applicable.

Why are the changes needed?

CHAR and VARCHAR extend StringType. As a result, the existing MapType(_: StringType, ...) pattern incorrectly accepts them as map key types, even though VariantGet.checkDataType explicitly rejects CHAR and VARCHAR in other positions. Since this validation is also used by to_variant_object, the same issue affects both variant conversion paths.

Does this PR introduce any user-facing change?

Yes. variant_get and to_variant_object now reject maps with CHAR or VARCHAR key types during analysis. Maps with STRING keys remain supported. CHAR and VARCHAR in array elements, map values, and struct fields were already rejected; the tests now cover these cases explicitly.

How was this patch tested?

Added unit tests covering positive STRING cases and negative CHAR/VARCHAR cases for variant_get and to_variant_object, including array elements, map keys/values, and struct fields.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex (GPT-5)

@ueshin

ueshin commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

cc @harshmotw-db @jovanm-db

@dongjoon-hyun dongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM

@ueshin ueshin changed the title [SPARK-58672][SQL] Reject CHAR/VARCHAR map keys in variant_get [SPARK-58672][SQL] Reject CHAR/VARCHAR map keys in variant conversions Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants