Skip to content

Comments

[SPARK-45162][SQL] Support maps and array parameters constructed via call_function#42894

Closed
MaxGekk wants to merge 6 commits intoapache:masterfrom
MaxGekk:fix-parameterized-sql-unresolved
Closed

[SPARK-45162][SQL] Support maps and array parameters constructed via call_function#42894
MaxGekk wants to merge 6 commits intoapache:masterfrom
MaxGekk:fix-parameterized-sql-unresolved

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Sep 12, 2023

What changes were proposed in this pull request?

In the PR, I propose to move the BindParameters rules from the Substitution to the Resolution batch, and change types of the args parameter of NameParameterizedQuery and PosParameterizedQuery to an Iterable to resolve argument expressions.

Why are the changes needed?

After the PR, the parameterized sql() allows map/array/struct constructed by functions like map(), array(), and struct(), but the same functions invoked via call_function are not supported:

scala> sql("SELECT element_at(:mapParam, 'a')", Map("mapParam" -> call_function("map", lit("a"), lit(1))))
org.apache.spark.sql.catalyst.ExtendedAnalysisException: [UNBOUND_SQL_PARAMETER] Found the unbound parameter: mapParam. Please, fix `args` and provide a mapping of the parameter to a SQL literal.; line 1 pos 18;

Does this PR introduce any user-facing change?

No, should not since it fixes an issue. Only if user code depends on the error message.

After the changes:

scala> sql("SELECT element_at(:mapParam, 'a')", Map("mapParam" -> call_function("map", lit("a"), lit(1)))).show(false)
+------------------------+
|element_at(map(a, 1), a)|
+------------------------+
|1                       |
+------------------------+

How was this patch tested?

By running new tests:

$ build/sbt "test:testOnly *ParametersSuite"

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the SQL label Sep 12, 2023
@MaxGekk MaxGekk changed the title [WIP][SQL] Support maps and array parameters constructed via call_function [WIP][SPARK-45162][SQL] Support maps and array parameters constructed via call_function Sep 14, 2023
@MaxGekk MaxGekk changed the title [WIP][SPARK-45162][SQL] Support maps and array parameters constructed via call_function [SPARK-45162][SQL] Support maps and array parameters constructed via call_function Sep 14, 2023
@MaxGekk MaxGekk marked this pull request as ready for review September 14, 2023 06:10
@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 14, 2023

@cloud-fan @peter-toth Could you review this PR, please.

@MaxGekk
Copy link
Member Author

MaxGekk commented Sep 14, 2023

Merging to master. Thank you, @cloud-fan for review.

@MaxGekk MaxGekk closed this in cd672b0 Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants