Skip to content

[CALCITE-3916] Implement top-down rule applying and upper bound space pruning - #1950

Closed
FatLittle wants to merge 4 commits into
apache:masterfrom
FatLittle:odps-cascade
Closed

FatLittle wants to merge 4 commits into
apache:masterfrom
FatLittle:odps-cascade

Conversation

@FatLittle

@FatLittle FatLittle commented Apr 28, 2020 •

Copy link
Copy Markdown

Link to https://issues.apache.org/jira/browse/CALCITE-3916

  1. Create a new CascadePlanner inherit from VolcanoPlanner
  2. Design tasks and task state transformations according to the design of Columnbia
  3. Introduce a LowerBoundCost metadata to compute lower bound and used for pruning

CheckInput(OptimizeInputs context,
RelNode parent, CascadeRelSubset input, int i, RelOptCost upper) {
this.context = context;
this.parent = parent;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any test for it?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet. Will add more.


/**
* An enum representing optimizing state of current RelSubset
*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between CascadeRelSubset and RelSubset?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CascadeRelSubset have two more fields: optimizeState and upperBound. The comments in this class will help you understand what they are used for.

@XuQianJin-Stars

Copy link
Copy Markdown
Contributor

hi, @FatLittle Thank you very much for the PR you submitted. Can you include the issue link in the description of this PR?

@FatLittle FatLittle closed this Apr 29, 2020
@FatLittle FatLittle reopened this Apr 29, 2020
@hsyuan

hsyuan commented Apr 29, 2020

Copy link
Copy Markdown
Member

@XuQianJin-Stars I think this is the most related JIRA:
https://issues.apache.org/jira/browse/CALCITE-3916

@FatLittle

Copy link
Copy Markdown
Author

@XuQianJin-Stars I think this is the most related JIRA:
https://issues.apache.org/jira/browse/CALCITE-3916

That's it. Thanks, Haisheng!

@FatLittle FatLittle changed the title Implement top-down rule applying and upper bound space pruning [CALCITE-3916] Implement top-down rule applying and upper bound space pruning Apr 29, 2020
@FatLittle
FatLittle force-pushed the odps-cascade branch 2 times, most recently from 1126e7b to 4d6e477 Compare May 9, 2020 05:48
protected boolean cascade() {
return false;
}

@chunweilei chunweilei May 9, 2020 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x

jinpeng.wjp added 2 commits May 11, 2020 12:36
@FatLittle
FatLittle force-pushed the odps-cascade branch 4 times, most recently from 9cfa69e to 54cde8d Compare May 11, 2020 10:01
@FatLittle

Copy link
Copy Markdown
Author

added a commit. Propagate Traits with Haisheng's trait propagation frameworks. Remove AC dependencies.

* CascadePlanner that apply memo pruning logic`
* based on the data structure of VolcanoPlanner
*/
public class CascadePlanner extends VolcanoPlanner {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the new planner's name should be CascadesPlanner not CascadePlanner according to the paper "The Cascades Framework for Query Optimization", and we can also find “Cascades” in the Columbia planner source code.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have modified the names in my new PR: #1991

}

RelOptCost lowerBound = null;
for (RelNode relNode : subset.getRels()) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If current group is not fully optimized and has no winner, there may be problems to get the lower bound here. Since the current group's optimization may affact the input groups of the current group. E.g., the current group's optimization may push down some require traits and maybe somewhere of the inputs subtrees may enhance there's costs. So if we get the lower bound here, it may be larger than the actual lower bound which may miss optimal plan

@hsyuan

hsyuan commented Jul 14, 2020

Copy link
Copy Markdown
Member

This PR is superseded by #1991.

@hsyuan hsyuan closed this Jul 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants