Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert to Delta SQL support #175

Closed
zsxwing opened this issue Sep 18, 2019 · 3 comments
Closed

Convert to Delta SQL support #175

zsxwing opened this issue Sep 18, 2019 · 3 comments
Labels
Milestone

Comments

@zsxwing
Copy link
Member

@zsxwing zsxwing commented Sep 18, 2019

Add the following SQL support for Convert to Delta:

CONVERT TO DELTA parquet.`path/to/table` [PARTITIONED BY (col_name1 col_type1, col_name2 col_type2, ...)]
@zsxwing zsxwing added the enhancement label Sep 18, 2019
@tdas tdas pinned this issue Sep 18, 2019
@cfmcgrady

This comment has been minimized.

Copy link

@cfmcgrady cfmcgrady commented Sep 20, 2019

hi, @zsxwing
spark SQL is doing support DML(delete/update) in the latest version, and I also find that delta is adding DML(describe/Convert/Vacuum) support now, is there any plan for add delete/update grammar support for spark-2.4.x?

@zsxwing

This comment has been minimized.

Copy link
Member Author

@zsxwing zsxwing commented Sep 20, 2019

@cfmcgrady we have looked at how to support merge/delete/update for Spark 2.4 in Delta Lake. However, we found out it would require to fork almost the whole Spark SQL parser and it would be pretty hard to maintain. Hence we decided to wait for Spark 3.0 instead. For Spark 2.4, we will just add several SQL utility commands (vacuum, describe history and convert to delta) to Delta Lake.

@tdas tdas unpinned this issue Oct 1, 2019
@mukulmurthy mukulmurthy added this to the Future Roadmap milestone Oct 2, 2019
@tdas tdas modified the milestones: Future Roadmap, 0.5.0 Nov 5, 2019
@pranavanand

This comment has been minimized.

Copy link
Contributor

@pranavanand pranavanand commented Nov 5, 2019

Completed as part of b624209

@tdas tdas closed this Nov 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.