Re-enable JoinPushTransitivePredicates optimization rule by default#18720
Merged
yashmayya merged 1 commit intoJun 10, 2026
Merged
Conversation
Reverts the default-disable from apache#17181. That rule was disabled because JoinPushTransitivePredicatesRule could spin in an expensive/looping predicate-inference path (RelMdPredicates#getPredicates). Calcite 1.42.0 (apache#18658) fixes this via CALCITE-7302 (calcite#4649), so the rule is safe to enable by default again. The lookup-join carve-out (no right-side pushdown) is preserved and covered by testJoinPushTransitivePredicateLookupJoin.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #18720 +/- ##
============================================
+ Coverage 64.48% 64.57% +0.08%
Complexity 1291 1291
============================================
Files 3372 3372
Lines 208639 208666 +27
Branches 32590 32606 +16
============================================
+ Hits 134550 134737 +187
+ Misses 63287 63102 -185
- Partials 10802 10827 +25
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
Author
|
Yeah the test exists in Calcite so I think we're good here - https://github.com/apache/calcite/blob/5bcecc1bdc9ce70086c1ebd9cf3dad7e22e2fec1/core/src/test/java/org/apache/calcite/test/RelOptRulesTest.java#L6796-L6807 |
cypherean
pushed a commit
to cypherean/pinot
that referenced
this pull request
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Re-enables the
JoinPushTransitivePredicatesplanner rule by default, reverting the default-disable introduced in #17181.Background
#17181 disabled this rule by default because
JoinPushTransitivePredicatesRulecould enter an expensive, effectively looping predicate-inference path during planning (RelMdPredicates#getPredicates→RelOptPredicateList.union→predicateConstants→gatherConstraints). In certain environments and query workloads this drove broker planning CPU from ~5% to ~70% alongside query timeouts. It was disabled as a default while remaining opt-in via theusePlannerRulesquery option.What changed
Calcite was upgraded to 1.42.0 in #18658, which includes the fix for CALCITE-7302 ("Infinite loop with
JoinPushTransitivePredicatesRule", apache/calcite#4649). That loop was caused by predicate-inference dedup failing once a conjunction was simplified into aSEARCH/Sargform, so the rule kept re-inferring the same predicates. With this fixed, the rule is safe to enable by default again.For reference, the earlier related fix CALCITE-6432 / apache/calcite#4346 (another
JoinPushTransitivePredicatesRuleinfinite-loop fix) shipped back in Calcite 1.40.0; CALCITE-7302 is the follow-up that 1.42.0 brings in.Behavior / blast radius
a.col1 = b.col1 AND a.col1 = 1, the rule also addsb.col1 = 1).QueryCompilationTest#testJoinPushTransitivePredicateLookupJoin).skipPlannerRules='JoinPushTransitivePredicates', or cluster-wide via thepinot.broker.mse.planner.disabled.rulesbroker config.Tests
ExplainPhysicalPlans.json,JoinPlans.json,PhysicalOptimizerPlans.json) to reflect the rule firing, and removed the now-unnecessaryusePlannerRulesopt-in fromQueryCompilationTest#testJoinPushTransitivePredicate.pinot-query-plannersuite passes (1263 tests, 0 failures).