Skip to content

[SPARK-37216][SQL] Add the Hive macro functionality to SparkSQL#34492

Closed
hgs19921112 wants to merge 4 commits intoapache:masterfrom
hgs19921112:master
Closed

[SPARK-37216][SQL] Add the Hive macro functionality to SparkSQL#34492
hgs19921112 wants to merge 4 commits intoapache:masterfrom
hgs19921112:master

Conversation

@hgs19921112
Copy link

What changes were proposed in this pull request?

Add the Hive macro functionality to SparkSQL

Why are the changes needed?

Some Hive sql can move to SparkSQL Smoothly

Does this PR introduce any user-facing change?

Some new DDL like 'create temparory macro ...'

How was this patch tested?

unit test

Authored-by: hgs19921112 haoguangshi@gmail.com

@github-actions github-actions bot added the SQL label Nov 5, 2021
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@hgs19921112 hgs19921112 changed the title [SPARK-37216] Add the Hive macro functionality to SparkSQL [SPARK-37216][SQL] Add the Hive macro functionality to SparkSQL Nov 5, 2021
@hgs19921112
Copy link
Author

Can anayone have a see? If this pr is useless , I will close it.

@xkrogen
Copy link
Contributor

xkrogen commented Nov 9, 2021

This seems to have some overlap with the recent functionality announced by Databricks (as an internal/proprietary feature): https://databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html
Is that understanding correct? Can you compare?
I am wondering if, similar to the Databricks feature, OSS Spark should have a more holistic way of handling this as opposed to copying Hive's syntax for a limited subset of the functionality.

cc @mridulm @wmoustafa @shardulm94

@hgs19921112
Copy link
Author

This seems to have some overlap with the recent functionality announced by Databricks (as an internal/proprietary feature): https://databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html Is that understanding correct? Can you compare? I am wondering if, similar to the Databricks feature, OSS Spark should have a more holistic way of handling this as opposed to copying Hive's syntax for a limited subset of the functionality.

cc @mridulm @wmoustafa @shardulm94
You are right . The functionality in Databricks's Spark is more powerful than this.I did not know about that before you mentioned it.Mybe this is excess.

'(' columns=multipartIdentifierPropertyList ')'
(OPTIONS options=propertyList)? #createIndex
| DROP INDEX (IF EXISTS)? identifier ON TABLE? multipartIdentifier #dropIndex
| CREATE TEMPORARY MACRO IDENTIFIER '(' colTypeList ')'
Copy link

@ever4Kenny ever4Kenny Aug 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be:
"| CREATE TEMPORARY MACRO IDENTIFIER '(' colTypeList? ')'"
to support no arg macros.
The colTypeList is optional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants