Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CALCITE-3916] Implement top-down rule applying and upper bound space pruning #1950

Closed
wants to merge 4 commits into from

Conversation

FatLittle
Copy link

@FatLittle FatLittle commented Apr 28, 2020

Link to https://issues.apache.org/jira/browse/CALCITE-3916

  1. Create a new CascadePlanner inherit from VolcanoPlanner
  2. Design tasks and task state transformations according to the design of Columnbia
  3. Introduce a LowerBoundCost metadata to compute lower bound and used for pruning

CheckInput(OptimizeInputs context,
RelNode parent, CascadeRelSubset input, int i, RelOptCost upper) {
this.context = context;
this.parent = parent;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any test for it?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet. Will add more.


/**
* An enum representing optimizing state of current RelSubset
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between CascadeRelSubset and RelSubset?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CascadeRelSubset have two more fields: optimizeState and upperBound. The comments in this class will help you understand what they are used for.

@XuQianJin-Stars
Copy link
Contributor

hi, @FatLittle Thank you very much for the PR you submitted. Can you include the issue link in the description of this PR?

@FatLittle FatLittle closed this Apr 29, 2020
@FatLittle FatLittle reopened this Apr 29, 2020
@hsyuan
Copy link
Member

hsyuan commented Apr 29, 2020

@XuQianJin-Stars I think this is the most related JIRA:
https://issues.apache.org/jira/browse/CALCITE-3916

@FatLittle
Copy link
Author

@XuQianJin-Stars I think this is the most related JIRA:
https://issues.apache.org/jira/browse/CALCITE-3916

That's it. Thanks, Haisheng!

@FatLittle FatLittle changed the title Implement top-down rule applying and upper bound space pruning [CALCITE-3916] Implement top-down rule applying and upper bound space pruning Apr 29, 2020
@FatLittle FatLittle force-pushed the odps-cascade branch 2 times, most recently from 1126e7b to 4d6e477 Compare May 9, 2020 05:48
protected boolean cascade() {
return false;
}

Copy link
Contributor

@chunweilei chunweilei May 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x

jinpeng.wjp added 2 commits May 11, 2020 12:36
@FatLittle FatLittle force-pushed the odps-cascade branch 4 times, most recently from 9cfa69e to 54cde8d Compare May 11, 2020 10:01
@FatLittle
Copy link
Author

added a commit. Propagate Traits with Haisheng's trait propagation frameworks. Remove AC dependencies.

* CascadePlanner that apply memo pruning logic`
* based on the data structure of VolcanoPlanner
*/
public class CascadePlanner extends VolcanoPlanner {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the new planner's name should be CascadesPlanner not CascadePlanner according to the paper "The Cascades Framework for Query Optimization", and we can also find “Cascades” in the Columbia planner source code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have modified the names in my new PR: #1991

}

RelOptCost lowerBound = null;
for (RelNode relNode : subset.getRels()) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If current group is not fully optimized and has no winner, there may be problems to get the lower bound here. Since the current group's optimization may affact the input groups of the current group. E.g., the current group's optimization may push down some require traits and maybe somewhere of the inputs subtrees may enhance there's costs. So if we get the lower bound here, it may be larger than the actual lower bound which may miss optimal plan

@hsyuan
Copy link
Member

hsyuan commented Jul 14, 2020

This PR is superseded by #1991.

@hsyuan hsyuan closed this Jul 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants