Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner,stats: don't re-calculate the index's ranges #12856

Merged
merged 9 commits into from Dec 4, 2019
Merged

planner,stats: don't re-calculate the index's ranges #12856

merged 9 commits into from Dec 4, 2019

Conversation

winoros
Copy link
Member

@winoros winoros commented Oct 21, 2019

What problem does this PR solve?

image

The things can be reused when calcuate ranges and selectivity.

What is changed and how it works?

Reuse them.

Check List

Tests

  • Exsiting Unit test

Code changes

  • Has exported function/method change
  • Has exported variable/fields change

Side effects

  • Increased code complexity

@winoros
Copy link
Member Author

winoros commented Oct 21, 2019

Actually, i want to split the codes which only do code moving as a single pr.
But there's something to discuss.
Basically, there're two ways to resolve the dependency cycle.
One is like what i did in this pr. Moving the AccessPath to planner/util. Then package statistics will rely on planner/util.
Another way is moving Selectivity to planner/core. So the caller of Selectivity will be in the same package with Selectivity.

Maybe the second way is better.

After we dicide the solution, i'll open a seperate pr to do it.

@winoros
Copy link
Member Author

winoros commented Oct 21, 2019

One explain test is changed. Being checking.

lzmhhh123
lzmhhh123 reviewed Oct 30, 2019
View reviewed changes
Copy link
Member

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any progress?

@winoros
Copy link
Member Author

winoros commented Oct 31, 2019

@lzmhhh123 There's a bug in statistics. I just located it.

winoros
winoros commented Oct 31, 2019
View reviewed changes
cmd/explaintest/r/explain_easy.result Outdated
├─IndexScan_8 33.33 cop[tikv] table:t1, index:c2, range:(1 1,1 +inf], keep order:false, stats:pseudo
└─Selection_10 1.11 cop[tikv] lt(Column#3, 1)
└─Selection_10 11.08 cop[tikv] lt(Column#3, 1)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is a bug related with statistics.
Since we take the RowID column into consideration when the column of index are all be used.
But when calculating the selecitivity. https://github.com/pingcap/tidb/blob/master/statistics/selectivity.go#L215 Here we doesn't consider the RowID column.
So the rowid column is calculated twice, which makes the final row count smaller.

I've tried but it's not very easy to fix. So we can make this pr merged first. Then i'll try to fix it in another pr.

@codecov
Copy link

codecov bot commented Oct 31, 2019 •
edited

Codecov Report

Merging #12856 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #12856   +/-   ##
===========================================
  Coverage   80.5393%   80.5393%           
===========================================
  Files           480        480           
  Lines        120926     120926           
===========================================
  Hits          97393      97393           
  Misses        15974      15974           
  Partials       7559       7559

alivxxx
alivxxx reviewed Oct 31, 2019
View reviewed changes
planner/util/path.go Show resolved Hide resolved
statistics/selectivity.go Outdated Show resolved Hide resolved
lzmhhh123
lzmhhh123 reviewed Oct 31, 2019
View reviewed changes
planner/util/path.go Show resolved Hide resolved
statistics/selectivity.go Outdated Show resolved Hide resolved
alivxxx
alivxxx reviewed Nov 5, 2019
View reviewed changes
Copy link
Contributor

@alivxxx alivxxx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
Please resolve the conflicts.

lzmhhh123
lzmhhh123 reviewed Nov 5, 2019
View reviewed changes
planner/util/path.go Outdated Show resolved Hide resolved
@winoros winoros requested a review from a team as a code owner December 4, 2019 08:34
@ghost ghost requested review from francis0407 and alivxxx and removed request for a team December 4, 2019 08:34
@winoros
Copy link
Member Author

winoros commented Dec 4, 2019

./run-all-tests

lzmhhh123
lzmhhh123 approved these changes Dec 4, 2019
View reviewed changes
Copy link
Member

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@winoros winoros merged commit 7b09a11 into pingcap:master Dec 4, 2019
@winoros winoros deleted the reduce-range-call branch December 4, 2019 09:19
XiaTianliang pushed a commit to XiaTianliang/tidb that referenced this pull request Dec 21, 2019
@winoros @XiaTianliang
planner,stats: don't re-calculate the index's ranges (pingcap#12856)
0df61f0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alivxxx alivxxx alivxxx left review comments

@lzmhhh123 lzmhhh123 lzmhhh123 approved these changes

Assignees
No one assigned
Labels
sig/planner SIG: Planner type/performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@winoros @alivxxx @lzmhhh123