Skip to content

[SPARK-56659][SQL] Show function resources in DESCRIBE FUNCTION output#55602

Open
TongWei1105 wants to merge 1 commit intoapache:masterfrom
TongWei1105:fix-describe-function-show-resources
Open

[SPARK-56659][SQL] Show function resources in DESCRIBE FUNCTION output#55602
TongWei1105 wants to merge 1 commit intoapache:masterfrom
TongWei1105:fix-describe-function-show-resources

Conversation

@TongWei1105
Copy link
Copy Markdown
Contributor

@TongWei1105 TongWei1105 commented Apr 29, 2026

What changes were proposed in this pull request?

When a persistent function is created with USING JAR/FILE resources, DESCRIBE FUNCTION does not display those resources in its output.

For example:

CREATE FUNCTION my_udf AS 'com.example.MyUDF' USING JAR 'hdfs:///path/to/my-udf.jar';
DESCRIBE FUNCTION my_udf;

Current output:
Function: spark_catalog.default.my_udf
Class: com.example.MyUDF
Usage: N/A

Expected output:
Function: spark_catalog.default.my_udf
Class: com.example.MyUDF
Usage: N/A
Resource: jar hdfs:///path/to/my-udf.jar

Why are the changes needed?

DescribeFunctionCommand receives ExpressionInfo during analysis, which does not carry FunctionResource metadata. Although the resources are persisted in the metastore via CatalogFunction, they are never retrieved
or displayed by the describe command. This makes it difficult for users to identify which JARs or files a function depends on.

Does this PR introduce any user-facing change?

Yes. DESCRIBE FUNCTION and DESCRIBE FUNCTION EXTENDED now display Resource: rows for persistent functions that were created with USING JAR/FILE/ARCHIVE clauses.

How was this patch tested?

Updated existing tests in SQLQuerySuite to verify that resource information is shown for persistent user-defined functions, covering both DESCRIBE FUNCTION and DESCRIBE FUNCTION EXTENDED.

Was this patch authored or co-authored using generative AI tooling?

Yes, Claude Opus 4.6

DescribeFunctionCommand only received ExpressionInfo which lacks
resource metadata, so USING JAR/FILE clauses were silently dropped
from the output. Fetch CatalogFunction from the metastore to display
resources for persistent functions.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@TongWei1105
Copy link
Copy Markdown
Contributor Author

cc @wangyum @cloud-fan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants