HIVE-28310: Disable hive.optimize.join.disjunctive.transitive.predicates.pushdown by default#5286
Closed
zabetak wants to merge 2 commits intoapache:masterfrom
Closed
HIVE-28310: Disable hive.optimize.join.disjunctive.transitive.predicates.pushdown by default#5286zabetak wants to merge 2 commits intoapache:masterfrom
zabetak wants to merge 2 commits intoapache:masterfrom
Conversation
zabetak
changed the title
asolimando
approved these changes
Jun 7, 2024
Member
asolimando
left a comment
asolimando
left a comment
There was a problem hiding this comment.
I agree that, given the number of escalations and issues we have seen over time due to disjunctive filters, it's safer to disable it by default.
Queries experiencing performance regressions can always turn this property on.
Approval modulo pending tests.
All these are expected regressions highlighting the impact that this change may have on actual queries. Observe that in terms of simplifications there are difficulties in the presence of negation/disjunction and the HiveJoinPushTransitivePredicatesRule. (UDFToDouble(key) NOT BETWEEN 20.0D AND 100.0D and (UDFToDouble(key) > 100.0D)) is equivalent to UDFToDouble(key) > 100.0D Some predicates that were transitively propagated were redundant (see plans before).
|
kasakrisz
approved these changes
Jun 28, 2024
mr3project
pushed a commit
to mr3project/hive-mr3
that referenced
this pull request
Aug 21, 2024
…tes.pushdown by default (Stamatis Zampetakis reviewed by Alessandro Solimando, Krisztian Kasa) Pushing dis-junction and can lead to OOM and HS2 crashes. Turning off this feature would makes HiveServer2 more stable at the expense of some less efficient plans. Close apache/hive#5286
dengzhhu653
pushed a commit
to dengzhhu653/hive
that referenced
this pull request
Sep 14, 2024
…tes.pushdown by default (Stamatis Zampetakis reviewed by Alessandro Solimando, Krisztian Kasa) Pushing dis-junction and can lead to OOM and HS2 crashes. Turning off this feature would makes HiveServer2 more stable at the expense of some less efficient plans. Close apache#5286
dengzhhu653
pushed a commit
to dengzhhu653/hive
that referenced
this pull request
Sep 20, 2024
…tes.pushdown by default (Stamatis Zampetakis reviewed by Alessandro Solimando, Krisztian Kasa) Pushing dis-junction and can lead to OOM and HS2 crashes. Turning off this feature would makes HiveServer2 more stable at the expense of some less efficient plans. Close apache#5286
mr3project
pushed a commit
to mr3project/hive-mr3
that referenced
this pull request
Nov 29, 2024
…tes.pushdown by default (Stamatis Zampetakis reviewed by Alessandro Solimando, Krisztian Kasa) Pushing dis-junction and can lead to OOM and HS2 crashes. Turning off this feature would makes HiveServer2 more stable at the expense of some less efficient plans. Close apache/hive#5286
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
Pushing dis-junction and can lead to OOM and HS2 crashes. Turning off this feature would make HS2 more stable at the expense of some less efficient plans.
Does this PR introduce any user-facing change?
Plan and potentially performance changes.
Is the change a dependency upgrade?
No
How was this patch tested?
Existing tests