Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner: support join elimination rule #8021

Merged
merged 29 commits into from Nov 9, 2018

Conversation

@lzmhhh123
Copy link
Member

commented Oct 23, 2018

What problem does this PR solve?

support join elimination rule #7704

What is changed and how it works?

Add a logical optimize rule:

  1. outer join elimination: For example left outer join, if the parent only use the columns from left table and the join key of right table(the inner table) is a unique key of the right table. the left outer join can be eliminated.
  2. outer join elimination with distinct: For example left outer join. If the parent only use the columns from left table with 'distinct' label. The left outer join can be eliminated.

Check List

Tests

  • Unit test
  • Integration test

Code changes

  • Has exported function/method change
  • Has exported variable/fields change

Side effects

  • Increased code complexity
lzmhhh123 added 6 commits Oct 23, 2018
@lzmhhh123

This comment has been minimized.

Copy link
Member Author

commented Oct 24, 2018

/run-all-tests

@lzmhhh123 lzmhhh123 removed the status/WIP label Oct 24, 2018
planner/core/optimizer.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
lzmhhh123 added 4 commits Oct 30, 2018
 into dev/join_elimination
@lzmhhh123

This comment has been minimized.

Copy link
Member Author

commented Oct 30, 2018

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
lzmhhh123 added 2 commits Nov 2, 2018
planner/core/optimizer.go Outdated Show resolved Hide resolved
lzmhhh123 and others added 2 commits Nov 2, 2018
@zz-jason

This comment has been minimized.

Copy link
Member

commented Nov 2, 2018

LGTM

@zz-jason zz-jason added the status/LGT1 label Nov 2, 2018
@zz-jason

This comment has been minimized.

Copy link
Member

commented Nov 2, 2018

planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
@XuHuaiyu

This comment has been minimized.

Copy link
Contributor

commented Nov 5, 2018

Will it be better to not store o.cols and o.schemas,
it seems we can put them as parameters?

planner/core/optimizer.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
}
}
if joinKeysContainIndex {
return p.children[1^innerChildIdx]

This comment has been minimized.

Copy link
@eurekaka

eurekaka Nov 6, 2018

Contributor

Should we remove other possible paths in this case?

planner/core/logical_plan_test.go Show resolved Hide resolved
lzmhhh123 added 2 commits Nov 6, 2018
@XuHuaiyu

This comment has been minimized.

Copy link
Contributor

commented Nov 6, 2018

Please check whether this would be more readable?

rule_join_elimination.txt

ci
planner/core/rule_join_elimination.go Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
planner/core/rule_join_elimination.go Outdated Show resolved Hide resolved
Copy link
Contributor

left a comment

LGTM

@XuHuaiyu

This comment has been minimized.

Copy link
Contributor

commented Nov 9, 2018

joinPlan.JoinType = LeftOuterJoin
resetNotNullFlag(joinPlan.schema, leftPlan.Schema().Len(), joinPlan.schema.Len())
case ast.RightJoin:
// right outer join need to be checked elimination
b.optFlag = b.optFlag | flagEliminateOuterJoin
joinPlan.JoinType = RightOuterJoin
resetNotNullFlag(joinPlan.schema, 0, leftPlan.Schema().Len())
default:

This comment has been minimized.

Copy link
@zz-jason

zz-jason Nov 9, 2018

Member

In the optimization phase, the semi joins are also considered, I think we should add the optimization flag for these semi joins in here.

This comment has been minimized.

Copy link
@zz-jason

zz-jason Nov 9, 2018

Member

and please add some explain tests to cover these optimizations.

This comment has been minimized.

Copy link
@lzmhhh123

lzmhhh123 Nov 9, 2018

Author Member

After my seriously considering. I think semi-joins can't be eliminated in any case. So I will remove semi-join in the eliminating rule.

fix
Copy link
Member

left a comment

LGTM

@zz-jason zz-jason added status/LGT2 and removed status/LGT1 labels Nov 9, 2018
@zz-jason

This comment has been minimized.

Copy link
Member

commented Nov 9, 2018

/run-all-tests

@zz-jason zz-jason merged commit cd5ffb3 into pingcap:master Nov 9, 2018
4 checks passed
4 checks passed
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci-tidb/build Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details
@lzmhhh123 lzmhhh123 assigned lzmhhh123 and unassigned lzmhhh123 Nov 9, 2018
@lzmhhh123 lzmhhh123 deleted the lzmhhh123:dev/join_elimination branch Nov 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.