Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner/core: separate `aggPrune` from `aggPushDown` #7676

Merged
merged 28 commits into from Oct 8, 2018

Conversation

Projects
None yet
4 participants
@winoros
Copy link
Member

commented Sep 12, 2018

What problem does this PR solve?

AggregationPushDown is closed by default since it's not always better. But AggregationElimination is always a good choice.

What is changed and how it works?

So separate it from aggPushDown

Check List

Tests

  • Unit test

Related changes

It's one part from #7531.

Release Note
reopen aggregation pruning optimization.

@eurekaka
Copy link
Contributor

left a comment

rest LGTM

Show resolved Hide resolved plan/optimizer.go Outdated
Show resolved Hide resolved plan/rule_aggregation_elimination.go Outdated
@winoros

This comment has been minimized.

Copy link
Member Author

commented Sep 12, 2018

Struct renaming changes will be pushed after merged #7680 if there's no need to change the code logic.

"github.com/pingcap/tidb/types"
)

type aggregationRecursiveEliminater struct {

This comment has been minimized.

Copy link
@lamxTyler

lamxTyler Sep 13, 2018

Member

Eliminator

@zz-jason

This comment has been minimized.

Copy link
Member

commented Sep 17, 2018

@winoros PTAL

winoros added some commits Sep 20, 2018

@winoros

This comment has been minimized.

Copy link
Member Author

commented Sep 20, 2018

Comments addressed.
PTAL @zz-jason @eurekaka @lamxTyler

@winoros

This comment has been minimized.

Copy link
Member Author

commented Sep 20, 2018

/run-all-tests

@lamxTyler
Copy link
Member

left a comment

LGTM

@zz-jason
Copy link
Member

left a comment

LGTM

@zz-jason

This comment has been minimized.

Copy link
Member

commented Sep 20, 2018

/run-common-test

@winoros

This comment has been minimized.

Copy link
Member Author

commented Sep 21, 2018

Seems there's precision error. I'm looking into it.

Waiting for fixing.

@winoros

This comment has been minimized.

Copy link
Member Author

commented Sep 21, 2018

Emmm... It seems that i need to reuse expression.aggregation.AggFuncDesc.typeInfer to make the rewriting works properly.

@winoros winoros force-pushed the winoros:extract-agg-prune branch from fb9b08d to 3842d88 Sep 26, 2018

@winoros

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2018

Now it works well.
PTAL @zz-jason @eurekaka @XuHuaiyu

@winoros winoros changed the title plan: separate `aggPrune` from `aggPushDown` planner/core: separate `aggPrune` from `aggPushDown` Sep 26, 2018

Show resolved Hide resolved expression/aggregation/descriptor.go
@@ -32,23 +32,25 @@ const (
flagPrunColumns uint64 = 1 << iota
flagEliminateProjection
flagBuildKeyInfo
flagEliminateAgg

This comment has been minimized.

Copy link
@zz-jason

zz-jason Sep 26, 2018

Member

how about moving it after flagDecorrelate?

This comment has been minimized.

Copy link
@winoros

winoros Sep 26, 2018

Author Member

Seems reasonable.

Show resolved Hide resolved planner/core/rule_aggregation_elimination.go
@@ -851,7 +851,7 @@ func (s *testPlanSuite) TestDAGPlanBuilderAgg(c *C) {
},
{
sql: "select (select count(1) k from t s where s.a = t.a having k != 0) from t",
best: "LeftHashJoin{TableReader(Table(t))->TableReader(Table(t)->StreamAgg)->StreamAgg->Sel([ne(k, 0)])}(test.t.a,s.a)->Projection->Projection",
best: "MergeLeftOuterJoin{TableReader(Table(t))->TableReader(Table(t))->Projection}(test.t.a,s.a)->Projection->Projection",

This comment has been minimized.

Copy link
@winoros

winoros Sep 27, 2018

Author Member

First decorrelate, it becomes select k from t left join (select count(1) k from t s group by a) on s.a=t.a and k != 0.
Then aggregate eliminate, it becomes select k from t left join (select if(isnull(1), 0, 1) k from t s) on s.a=t.a and k != 0. Since if(isnull(1), 0, 1) is always 1. So it simplified as select k from t left join (select 1 k from t s) on s.a=t.a and k != 0.
Finally predicate push down, k != 0 pushed down and converted to 1 != 0 which is always true.
So Sel([ne(k, 0)]) is removed after that aggregate eliminate is operated after decorrelate.

winoros and others added some commits Sep 27, 2018

@zz-jason

This comment has been minimized.

Copy link
Member

commented Sep 27, 2018

/run-all-tests

@winoros

This comment has been minimized.

Copy link
Member Author

commented Sep 27, 2018

/run-common-test tidb-test=pr/631
/run-integration-common-test tidb-test=pr/631

@winoros

This comment has been minimized.

Copy link
Member Author

commented Sep 27, 2018

/run-common-test tidb-test=pr/631

@zz-jason

This comment has been minimized.

Copy link
Member

commented Sep 27, 2018

LGTM

@zz-jason

This comment has been minimized.

Copy link
Member

commented Sep 27, 2018

@winoros winoros added the status/LGT1 label Sep 27, 2018

@eurekaka
Copy link
Contributor

left a comment

LGTM

@eurekaka eurekaka added status/LGT2 and removed status/LGT1 labels Oct 8, 2018

eurekaka and others added some commits Oct 8, 2018

@winoros winoros merged commit 78303cb into pingcap:master Oct 8, 2018

4 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci-tidb/build Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@winoros winoros deleted the winoros:extract-agg-prune branch Oct 8, 2018

XuHuaiyu added a commit to XuHuaiyu/tidb that referenced this pull request Oct 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.