Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Spark][Databricks] DropTableVisitor throwing an exception #2544

Open
pawel-big-lebowski opened this issue Mar 28, 2024 · 0 comments
Open

[Spark][Databricks] DropTableVisitor throwing an exception #2544

pawel-big-lebowski opened this issue Mar 28, 2024 · 0 comments
Labels
area:integration/spark source:astronomer Originates from the Astronomer team or tooling tool:databricks Databricks

Comments

@pawel-big-lebowski
Copy link
Contributor

DropTableVisitor assumes a child of Spark logical plan node is ResolvedTable:

ResolvedTable resolvedTable = ((ResolvedTable) (dropTable).child());

I observed that on Databricks runtime 14.2 (spark 3.5), this assumption no longer holds and ResolvedIdentifier is returned instead.
@pawel-big-lebowski pawel-big-lebowski mentioned this issue Mar 28, 2024
Merged
9 tasks
@pawel-big-lebowski pawel-big-lebowski changed the title [SPARK][DATABRICKS] DropTableVisitor throwing an exception [Spark][Databricks] DropTableVisitor throwing an exception Mar 28, 2024
@merobi-hub merobi-hub added the source:astronomer Originates from the Astronomer team or tooling label May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:integration/spark source:astronomer Originates from the Astronomer team or tooling tool:databricks Databricks
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pawel-big-lebowski @merobi-hub