Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Spark SQL (or HQL USING clause) #117

Closed
jpugliesi opened this issue Feb 22, 2022 · 2 comments
Closed

Support Spark SQL (or HQL USING clause) #117

jpugliesi opened this issue Feb 22, 2022 · 2 comments

Comments

@jpugliesi
Copy link

Is your feature request related to a problem? Please describe.
Spark SQL DDL is very similar to HQL, with a notable difference being that Spark SQL supports a USING clause in CREATE TABLE statements (see the DDL docs here).

The USING clause is importantly used for table definition in many storage formats provide APIs for Spark - for example, Apache Iceberg and Delta Lake

Describe the solution you'd like
Support Spark SQL DDL parsing, likely as a simple extension of HQL, with support for the USING clause

Describe alternatives you've considered
Potentially just support a USING clause in the HQL dialect parser

@xnuinside
Copy link
Owner

@jpugliesi hi, I saw issue, sorry for the long answer and thanks for reporting this. But I sure that I cannot work on it in the nearest week. I think you know that is going on with Putins army.. we tries to leave country with family, I hope 2 weeks later I can work on it. sorry for the delay

@xnuinside
Copy link
Owner

It was hard month... I just released v0.26.0 - https://pypi.org/project/simple-ddl-parser/ and it contains support for USING statement. Test sample: https://github.com/xnuinside/simple-ddl-parser/blob/main/tests/test_spark_sql.py#L6 if you will find any statements that does not work correctly - feel free to open the new issue @jpugliesi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants