-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-7824][SQL]Collapsing operator reordering and constant folding into a single batch to push down the single side. #6351
Closed
Closed
Changes from all commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
c3f046f
Merge pull request #1 from apache/master
pzzs cb1852d
Merge pull request #2 from apache/master
pzzs c87e8b6
Merge pull request #3 from apache/master
pzzs 161cae3
Merge pull request #4 from apache/master
pzzs 98b134f
Merge pull request #5 from apache/master
pzzs d00303b
Merge pull request #6 from apache/master
pzzs 802261c
Merge pull request #7 from apache/master
pzzs 34b1a9a
Merge pull request #8 from apache/master
pzzs f61210c
Merge pull request #9 from apache/master
pzzs f12fa50
Merge pull request #10 from apache/master
pzzs f03fe7f
Merge pull request #12 from apache/master
pzzs 14952e2
Merge pull request #13 from apache/master
pzzs ab8e9a6
Merge pull request #14 from apache/master
pzzs fa65718
Update Optimizer.scala
pzzs d98bc35
Update FilterPushdownSuite.scala
pzzs 8df716a
Update FilterPushdownSuite.scala
pzzs b01e622
Merge pull request #15 from apache/master
pzzs be6b1d5
Update Optimizer.scala
pzzs f2ee5fe
Update Optimizer.scala
pzzs 11beb61
Update FilterPushdownSuite.scala
pzzs a04ffae
Update Optimizer.scala
pzzs ae3af6d
Update FilterPushdownSuite.scala
pzzs c529d9f
Update FilterPushdownSuite.scala
pzzs f8b9314
Update FilterPushdownSuite.scala
pzzs 0ba5f42
Update Optimizer.scala
pzzs 7bc7d28
Merge pull request #17 from apache/master
pzzs 20de7be
Update Optimizer.scala
pzzs File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -38,21 +38,20 @@ object DefaultOptimizer extends Optimizer { | |
EliminateSubQueries) :: | ||
Batch("Distinct", FixedPoint(100), | ||
ReplaceDistinctWithAggregate) :: | ||
Batch("Operator Reordering", FixedPoint(100), | ||
Batch("Operator Optimizations", FixedPoint(100), | ||
UnionPushdown, | ||
CombineFilters, | ||
PushPredicateThroughProject, | ||
PushPredicateThroughJoin, | ||
PushPredicateThroughGenerate, | ||
ColumnPruning, | ||
ProjectCollapsing, | ||
CombineLimits) :: | ||
Batch("ConstantFolding", FixedPoint(100), | ||
CombineLimits, | ||
NullPropagation, | ||
OptimizeIn, | ||
ConstantFolding, | ||
LikeSimplification, | ||
BooleanSimplification, | ||
PushPredicateThroughJoin, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we don't need to change the rule order inside a batch with fixed point. Rules within a single batch shouldn't be sensitive to execution order. |
||
SimplifyFilters, | ||
SimplifyCasts, | ||
SimplifyCaseConversionExpressions) :: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about "Operator Reordering and ConstantFolding"