release-19.2: sql: only include the number of non-null rows when building histograms #48645
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Histograms used by the optimizer are built by sampling at most 10,000 rows, splitting those rows into 200 buckets, and then scaling up the counts of each bucket based on the total number of rows in the table. Prior to this commit, null values in the sampled column were excluded from the sampled values but included in the row count used to scale up the bucket counts. This could result in inaccurate histograms when there were many nulls. This commit fixes the problem by excluding nulls from the row count used to scale up the histogram bucket counts. Release note (performance improvement): Histograms used by the optimizer for query planning now have more accurate row counts per histogram bucket, particularly for columns that have many null values. This results in better plans in some cases.
|
This change is |
RaduBerinde
approved these changes
May 10, 2020
|
bors r+ |
|
bors r+ |
Build failed |
|
Flaked on #47377 bors r+ |
Build succeeded |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport 1/3 commits from #48528.
/cc @cockroachdb/release
Histograms used by the optimizer are built by sampling at most 10,000
rows, splitting those rows into 200 buckets, and then scaling up the
counts of each bucket based on the total number of rows in the table.
Prior to this commit, null values in the sampled column were excluded
from the sampled values but included in the row count used to scale
up the bucket counts. This could result in inaccurate histograms when
there were many nulls.
This commit fixes the problem by excluding nulls from the row count used
to scale up the histogram bucket counts.
Release note (performance improvement): Histograms used by the optimizer
for query planning now have more accurate row counts per histogram bucket,
particularly for columns that have many null values. This results in
better plans in some cases.