Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

release-19.2: sql: only include the number of non-null rows when building histograms #48645

Merged
merged 1 commit into from May 10, 2020
Merged

release-19.2: sql: only include the number of non-null rows when building histograms #48645

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
35 changes: 35 additions & 0 deletions pkg/sql/logictest/testdata/logic_test/distsql_stats
View file
Open in desktop
Expand Up @@ -263,6 +263,41 @@ default_stat2 {rowid} 1 1 0
default_stat2 {y} 1 1 1
default_stat2 {x} 1 1 1

# Add a few more rows.
statement ok
INSERT INTO simple VALUES (DEFAULT, DEFAULT);
INSERT INTO simple VALUES (0, DEFAULT);
INSERT INTO simple VALUES (DEFAULT, 0);
INSERT INTO simple VALUES (0, 1);

# Add an index.
statement ok
CREATE INDEX ON simple (x, y)

statement ok
CREATE STATISTICS default_stat3 FROM simple

query TTIII colnames
SELECT statistics_name, column_names, row_count, distinct_count, null_count
FROM [SHOW STATISTICS FOR TABLE simple]
----
statistics_name column_names row_count distinct_count null_count
default_stat3 {rowid} 5 5 0
default_stat3 {y} 5 3 3
default_stat3 {x} 5 2 3

let $hist_id_3
SELECT histogram_id FROM [SHOW STATISTICS FOR TABLE simple]
WHERE statistics_name = 'default_stat3' AND column_names = '{y}'

# The counts in each bucket should not include null values.
query TIRI colnames
SHOW HISTOGRAM $hist_id_3
----
upper_bound range_rows distinct_range_rows equal_rows
0 0 0 1
1 0 0 1

#
# Test numeric references
#
Expand Down
5 changes: 3 additions & 2 deletions pkg/sql/rowexec/sample_aggregator.go
View file
Open in desktop
Expand Up @@ -337,7 +337,7 @@ func (s *sampleAggregator) writeResults(ctx context.Context) error {
s.sr.Get(),
colIdx,
typ,
si.numRows,
si.numRows-si.numNulls,
distinctCount,
int(si.spec.HistogramMaxBuckets),
)
Expand Down Expand Up @@ -394,7 +394,8 @@ func (s *sampleAggregator) writeResults(ctx context.Context) error {

// generateHistogram returns a histogram (on a given column) from a set of
// samples.
// numRows is the total number of rows from which values were sampled.
// numRows is the total number of rows from which values were sampled
// (excluding rows that have NULL values on the histogram column).
func (s *sampleAggregator) generateHistogram(
ctx context.Context,
evalCtx *tree.EvalContext,
Expand Down
3 changes: 2 additions & 1 deletion pkg/sql/stats/histogram.go
View file
Open in desktop
Expand Up @@ -34,7 +34,8 @@ var HistogramClusterMode = settings.RegisterBoolSetting(
// the same number of samples (though it can vary when a boundary value has
// high frequency).
//
// numRows is the total number of rows from which values were sampled.
// numRows is the total number of rows from which values were sampled
// (excluding rows that have NULL values on the histogram column).
//
// In addition to building the histogram buckets, EquiDepthHistogram also
// estimates the number of distinct values in each bucket. It distributes the
Expand Down