Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner: consider disk cost in hashJoin #13246

Merged
merged 10 commits into from
Dec 3, 2019
Merged

planner: consider disk cost in hashJoin #13246

merged 10 commits into from
Dec 3, 2019

Conversation

SunRunAway
Copy link
Contributor

What problem does this PR solve?

part of #11607

What is changed and how it works?

Check List

Tests

  • Unit test

@SunRunAway SunRunAway mentioned this pull request Nov 7, 2019
Open
14 tasks
@codecov
Copy link

codecov bot commented Nov 19, 2019 •
edited

Codecov Report

Merging #13246 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #13246   +/-   ##
===========================================
  Coverage   80.3773%   80.3773%           
===========================================
  Files           474        474           
  Lines        118353     118353           
===========================================
  Hits          95129      95129           
  Misses        15804      15804           
  Partials       7420       7420

@SunRunAway SunRunAway force-pushed the disk-hash-phy-plan branch 3 times, most recently from 81ecec9 to 57c889d Compare November 21, 2019 08:34
@SunRunAway
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Nov 23, 2019

@eurekaka, @qw4990, @XuHuaiyu, @lzmhhh123, PTAL.

francis0407
francis0407 reviewed Nov 25, 2019
View reviewed changes
planner/core/task.go Outdated
Comment on lines 414 to 416
if (p.InnerChildIdx == 1 && !p.UseOuterToBuild) || (p.InnerChildIdx == 0 && p.UseOuterToBuild) {
innerCnt, outerCnt = rCnt, lCnt
inner = p.children[1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name inner might be confusing here. When p.InnerChildIdx == 0 && p.UseOuterToBuild, we would set the outer table as inner. Can we use something like build or stream here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed.

lzmhhh123
lzmhhh123 reviewed Nov 25, 2019
View reviewed changes
Copy link
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qw4990 qw4990 added the status/LGT1 Indicates that a PR has LGTM 1. label Nov 26, 2019
XuHuaiyu
XuHuaiyu reviewed Nov 26, 2019
View reviewed changes
planner/core/task.go
return cpuCost + memoryCost

if spill {
memoryCost *= float64(memQuota) / (rowSize * buildCnt)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does these code can describe the meaning?

memUsed := float64(memQuota)
originMemUsed := rowSize * buildCnt
factor := memUsed / originMemUsed
memoryCost *= factor

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got you, I misunderstood it before.
I think memoryCost *= float64(memQuota) / (rowSize * buildCnt) is ok.

@XuHuaiyu
Copy link
Contributor

LGTM

@XuHuaiyu
Copy link
Contributor

PTAL @eurekaka

lzmhhh123
lzmhhh123 previously approved these changes Nov 26, 2019
View reviewed changes
@lzmhhh123 lzmhhh123 added status/can-merge Indicates a PR has been approved by a committer. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Nov 26, 2019
@SunRunAway
Copy link
Contributor Author

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Dec 3, 2019

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Dec 3, 2019

@SunRunAway merge failed.

@SunRunAway
Copy link
Contributor Author

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Dec 3, 2019

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Dec 3, 2019

@SunRunAway merge failed.

@SunRunAway SunRunAway added status/can-merge Indicates a PR has been approved by a committer. and removed status/can-merge Indicates a PR has been approved by a committer. labels Dec 3, 2019
@glorv
Copy link
Contributor

glorv commented Dec 3, 2019

/build

@glorv
Copy link
Contributor

glorv commented Dec 3, 2019

/run-all-tests

@zz-jason
Copy link
Member

zz-jason commented Dec 3, 2019

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Dec 3, 2019

Your auto merge job has been accepted, waiting for 13863, 13859

@sre-bot sre-bot requested a review from a team as a code owner December 3, 2019 05:09
@ghost ghost requested review from zz-jason and removed request for a team December 3, 2019 05:10
@sre-bot
Copy link
Contributor

sre-bot commented Dec 3, 2019

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Dec 3, 2019

/run-all-tests

@SunRunAway SunRunAway mentioned this pull request Dec 3, 2019
Closed
@SunRunAway
Copy link
Contributor Author

/run-unit-test

@SunRunAway SunRunAway merged commit 5a589c9 into pingcap:master Dec 3, 2019
@SunRunAway SunRunAway deleted the disk-hash-phy-plan branch December 3, 2019 05:43
XiaTianliang pushed a commit to XiaTianliang/tidb that referenced this pull request Dec 21, 2019
@SunRunAway @XiaTianliang
planner: consider disk cost in hashJoin (pingcap#13246)
fdb8ced
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@XuHuaiyu XuHuaiyu XuHuaiyu left review comments

@francis0407 francis0407 francis0407 left review comments

@eurekaka eurekaka eurekaka approved these changes

@lzmhhh123 lzmhhh123 lzmhhh123 left review comments

Assignees
No one assigned
Labels
sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/new-feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
9 participants
@SunRunAway @sre-bot @XuHuaiyu @eurekaka @glorv @zz-jason @francis0407 @lzmhhh123 @qw4990