[SPARK-39880][SQL] V2 SHOW FUNCTIONS command should print qualified function name like v1 #37301
Conversation
|
cc @MaxGekk |
| } | ||
| } | ||
|
|
||
| test("show a function by its id") { |
There was a problem hiding this comment.
Removing it because it's almost the same as show a function in the USER name space
| override protected def createFunction(name: String): Unit = { | ||
| spark.udf.register(name, (arg1: Int, arg2: String) => arg2 + arg1) | ||
| sql(s"CREATE FUNCTION $name AS '${classOf[MyDoubleSum].getName}'") |
There was a problem hiding this comment.
V1 in memory catalog supports persistent functions as well: the UDAF interface.
| } else Seq.empty | ||
| val allFunctions = StringUtils.filterPattern( | ||
| userFunctions ++ systemFunctions, | ||
| pattern.getOrElse("*")).distinct.sorted |
There was a problem hiding this comment.
In my minor opinion, the origin code only call StringUtils.filterPattern once, it seems no need create a new method applyPattern.
There was a problem hiding this comment.
We can't apply only once after this PR. The pattern should be applied to function names only, not qualified names. So I have to apply the pattern separately for persistent functions before generating qualified names.
There was a problem hiding this comment.
I see. function names has been qualified.
|
+1, LGTM. Merging to master. |
What changes were proposed in this pull request?
This PR fixes a mismatch between v1 and v2 SHOW FUNCTIONS command: v1 prints qualified function names such as
spark_catalog.db1.f1, but v2 only prints the function name. V2 command should follow v1 behavior which makes more sense.This PR also updates the SHOW FUNCTIONS test suites to match the new behavior.
Why are the changes needed?
keep v1 and v2 commands consistent.
Does this PR introduce any user-facing change?
No, v2 SHOW FUNCTIONS is new in Spark 3.4 (not released)
How was this patch tested?
updated tests