Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Node as isDeleteTarget argument #6296

Merged
merged 1 commit into from
Dec 11, 2020

Conversation

losipiuk
Copy link
Member

@losipiuk losipiuk commented Dec 11, 2020

Fixes: #6294
Replaces: #6288

As a followup we probably should pursue an approach to store Analysis.target as a NodeRef instead QualifiedObjectName. There is a chance that a single query contains a table that is a delete target more than once. We should not treat all those occurrences as delete target.

@losipiuk losipiuk removed the WIP label Dec 11, 2020
@sopel39
Copy link
Member

sopel39 commented Dec 11, 2020

make sure all tests pass (I think complex delete is tested also in raptor)

@losipiuk losipiuk force-pushed the lo/is-delete-target-node branch 2 times, most recently from 099cf73 to 98685f4 Compare December 11, 2020 11:14
@losipiuk
Copy link
Member Author

@martint We probably want this one for 348 still.

@losipiuk
Copy link
Member Author

make sure all tests pass (I think complex delete is tested also in raptor)

Sure.

Any proposals for extra coverage to be added are welcome.

Copy link
Member

@findepi findepi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

This fixes two problems:

  • VIEW that contains unqalified names inside

    • thanks for the test with Presto VIEW
    • please add tests with Hive VIEW as well, to rule out any other implications -- this bug is fixed, but it uncovered general gap in our testing of views
  • DELERE FROM t that uses t in subquery as well

    • please add a test for that as well (i think there might have been no end-user visible effect of the bug ("innocent bug") though)

@kasiafi
Copy link
Member

kasiafi commented Dec 11, 2020

DELERE FROM t that uses t in subquery as well
please add a test for that as well (i think there might have been no end-user visible effect of the bug ("innocent bug") though)

I don't think this is addressed as long as Analysis performs the check by QualifiedObjectName and not by NodeRef. That requires recording target by NodeRef, as @losipiuk suggested.

i think there might have been no end-user visible effect of the bug

The only effect so far of tagging a TableScanNode incorrectly as the delete target is that a certain optimization is disabled in that branch of plan (namely, the rewrite of filtering semi-join to inner). However, the flag forDelete was meant to be used to disable other optimizations that are not relevant in the context of delete (e.g. join flipping).

I think that not taking this flag into consideration is a more serious issue than setting it where it doesn't belong.

@findepi
Copy link
Member

findepi commented Dec 11, 2020

I think that not taking this flag into consideration is a more serious issue than setting it where it doesn't belong.

Today. But this could change in the future.

Would it be possible to have a validator that verifies the flag is properly set and retained during optimizations?
We could also verify it is not set for ineligible table scans.
(of course, the second check is only possible after the Analysis changes as proposed)

(i don't mean the validator should be part of this PR)

@@ -98,6 +98,7 @@
private final Statement root;
private final Map<NodeRef<Parameter>, Expression> parameters;
private String updateType;
// TODO Reference by NodeRef instead name so we can distinguish actual target from other references to same table in the query
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should fix this, not just leave a TODO. Otherwise, any query such as DELETE FROM t WHERE t.v IN (SELECT ... FROM t WHERE ...) will do the wrong thing. That's actually a regression from how this used to work before isDeleteTarget was added.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@martint yes, we discussed that.
We want to fix the user-visible bug for views first (this PR)
and i then the (currently user-invisible) bug will be follow up

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's actually a regression from how this used to work before isDeleteTarget was added.

Currently this flag prevents semi_join->join rewrite, so when it's true for target table scans, previous behavior is preserved.

@sopel39
Copy link
Member

sopel39 commented Dec 11, 2020

@losipiuk you mixed up commits from other PR

@losipiuk
Copy link
Member Author

@losipiuk you mixed up commits from other PR

thanks :)

@sopel39 sopel39 added the bug Something isn't working label Dec 11, 2020
@sopel39 sopel39 merged commit 4f02e77 into trinodb:master Dec 11, 2020
@sopel39 sopel39 mentioned this pull request Dec 11, 2020
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cla-signed
Development

Successfully merging this pull request may close these issues.

Referencing VIEW fails if session catalog & schema are not set
5 participants