Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql/opt: add variables to disable statistics forecasting #86350

Closed
michae2 opened this issue Aug 17, 2022 · 0 comments · Fixed by #86932
Closed

sql/opt: add variables to disable statistics forecasting #86350

michae2 opened this issue Aug 17, 2022 · 0 comments · Fixed by #86932
Assignees
@michae2
Labels
A-sql-optimizer SQL logical planning and optimizations. A-sql-table-stats Table statistics (and their automatic refresh). C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team
Projects
SQL Queries

Comments

@michae2
Copy link
Collaborator

michae2 commented Aug 17, 2022 •
edited by cockroach-jira-scripts

Follow up from #79872: now that statistics forecasts are enabled for all tables, we need a session variable that can disable usage of forecasts in the optimizer.

Jira issue: CRDB-18707
@michae2 michae2 added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-sql-optimizer SQL logical planning and optimizations. A-sql-table-stats Table statistics (and their automatic refresh). T-sql-queries SQL Queries Team labels Aug 17, 2022
@michae2 michae2 self-assigned this Aug 17, 2022
@blathers-crl blathers-crl bot added this to Triage in SQL Queries Aug 17, 2022
@michae2 michae2 mentioned this issue Aug 17, 2022
Closed
@mgartner mgartner moved this from Triage to Up Next in SQL Queries Aug 18, 2022
@rytaft rytaft moved this from Up Next to Active in SQL Queries Aug 23, 2022
@michae2 michae2 mentioned this issue Aug 24, 2022
Merged
michae2 added a commit to michae2/cockroach that referenced this issue Aug 25, 2022
@michae2
sql/opt: add session variable to disable stats forecast use in optimizer
67f9187 
Add a new session variable, `optimizer_use_forecasts`, which can be used
to disable forecast usage. Forecasts will still be generated in the
stats cache (this will be controlled by a different variable).

Assists: cockroachdb#86350

Release justification: Low-risk update to new functionality.

Release note (sql change): This commit adds a new session variable,
`optimizer_use_forecasts`, which can be set to false to disable usage of
statistics forecasts when optimizing a query.
michae2 added a commit to michae2/cockroach that referenced this issue Aug 26, 2022
@michae2
sql/opt: add session setting to disable stats forecast use in optimizer
4b2b1a2 
Add a new session setting, `optimizer_use_forecasts`, which can be used
to disable forecast usage in the optimizer. Forecasts will still be
generated in the stats cache (this will be controlled by a different
setting).

Assists: cockroachdb#86350

Release justification: Low-risk update to new functionality.

Release note (sql change): This commit adds a new session setting,
`optimizer_use_forecasts`, which can be set to false to disable usage of
statistics forecasts when optimizing a query.
michae2 added a commit to michae2/cockroach that referenced this issue Aug 26, 2022
@michae2
sql/stats: add cluster setting to disable generation of stats forecasts
76b22ec 
Add a new cluster setting `sql.stats.forecasts.enabled` that can be
used to disable generation of statistics forecasts in the stats cache.
Changing this setting will cause the stats cache to gradually evict and
reload all cached statistics as they are requested.

Fixes: cockroachdb#86350

Release justification: Low-risk update to new functionality.

Release note (sql change): Add a new cluster setting
`sql.stats.forecasts.enabled` which controls whether statistics
forecasts are generated by default for all tables. Note that this is
different from the session setting `optimizer_use_forecasts` which
controls whether statistics forecasts are used by the current session.
@michae2 michae2 mentioned this issue Aug 26, 2022
Merged
michae2 added a commit to michae2/cockroach that referenced this issue Aug 26, 2022
@michae2
sql/opt: add session setting to disable stats forecast use in optimizer
3eea6fa 
Add a new session setting, `optimizer_use_forecasts`, which can be used
to disable forecast usage in the optimizer. Forecasts will still be
generated in the stats cache (this will be controlled by a different
setting).

Assists: cockroachdb#86350

Release justification: Low-risk update to new functionality.

Release note (sql change): This commit adds a new session setting,
`optimizer_use_forecasts`, which can be set to false to disable usage of
statistics forecasts when optimizing a query.
michae2 added a commit to michae2/cockroach that referenced this issue Aug 26, 2022
@michae2
sql/stats: add cluster setting to disable generation of stats forecasts
26e0686 
Add a new cluster setting `sql.stats.forecasts.enabled` that can be
used to disable generation of statistics forecasts in the stats cache.
Changing this setting will cause the stats cache to gradually evict and
reload all cached statistics as they are requested.

Fixes: cockroachdb#86350

Release justification: Low-risk update to new functionality.

Release note (sql change): Add a new cluster setting
`sql.stats.forecasts.enabled` which controls whether statistics
forecasts are generated by default for all tables. Note that this is
different from the session setting `optimizer_use_forecasts` which
controls whether statistics forecasts are used by the current session.
michae2 added a commit to michae2/cockroach that referenced this issue Aug 26, 2022
@michae2
sql/stats: add cluster setting to disable generation of stats forecasts
ea3cc4e 
Add a new cluster setting `sql.stats.forecasts.enabled` that can be
used to disable generation of statistics forecasts in the stats cache.
Changing this setting will cause the stats cache to gradually evict and
reload all cached statistics as they are requested.

Fixes: cockroachdb#86350

Release justification: Low-risk update to new functionality.

Release note (sql change): Add a new cluster setting
`sql.stats.forecasts.enabled` which controls whether statistics
forecasts are generated by default for all tables. (Note that this is
different from the session setting `optimizer_use_forecasts` which
controls whether statistics forecasts are used when optimizing the
current query. If forecasting is disabled, then even if
`optimizer_use_forecasts` is true for a given query it won't have any
forecasts to use.)
craig bot pushed a commit that referenced this issue Aug 26, 2022
@michae2 @yuzefovich @srosenberg
Merge #86834 #86920 #86954 #86956
8e3b269 
86834: sql/opt: add session setting to disable stats forecast use in optimizer r=rytaft a=michae2

Add a new session setting, `optimizer_use_forecasts`, which can be used
to disable forecast usage in the optimizer. Forecasts will still be
generated in the stats cache (this will be controlled by a different
setting).

Assists: #86350

Release justification: Low-risk update to new functionality.

Release note (sql change): This commit adds a new session setting,
`optimizer_use_forecasts`, which can be set to false to disable usage of
statistics forecasts when optimizing a query.

86920: roachprod{,-stress}: let roachprod-stress use roachprod library r=srosenberg a=healthy-pod

This patch migrates roachprod-stress from using the roachprod
binary to using the roachprod library.

Release justification: Non-production code changes
Release note: None

86954: util/mon: remove nameWithPointer to reduce allocations r=yuzefovich a=yuzefovich

This field was added in order to help us track down some of the memory
leaks which we have already found, and the field didn't turn out to be
that useful. When it was introduced, the implications on the increase in
allocations were unknown, and now I don't think the field is worth it.

```
name                                           old time/op    new time/op    delta
FlowSetup/vectorize=true/distribute=true-24       168µs ± 5%     164µs ± 4%  -2.33%  (p=0.007 n=19+20)
FlowSetup/vectorize=true/distribute=false-24      167µs ± 6%     164µs ± 6%    ~     (p=0.060 n=20+20)
FlowSetup/vectorize=false/distribute=true-24      163µs ± 4%     161µs ± 7%    ~     (p=0.057 n=19+20)
FlowSetup/vectorize=false/distribute=false-24     161µs ± 6%     159µs ± 5%    ~     (p=0.309 n=19+20)

name                                           old alloc/op   new alloc/op   delta
FlowSetup/vectorize=true/distribute=true-24      19.6kB ± 8%    19.0kB ± 8%  -2.62%  (p=0.001 n=19+18)
FlowSetup/vectorize=true/distribute=false-24     18.2kB ± 1%    17.7kB ± 1%  -2.56%  (p=0.000 n=17+16)
FlowSetup/vectorize=false/distribute=true-24     25.8kB ± 2%    25.4kB ± 0%  -1.44%  (p=0.000 n=16+16)
FlowSetup/vectorize=false/distribute=false-24    24.7kB ± 0%    24.4kB ± 1%  -1.36%  (p=0.000 n=16+16)

name                                           old allocs/op  new allocs/op  delta
FlowSetup/vectorize=true/distribute=true-24         218 ± 2%       205 ± 3%  -5.64%  (p=0.000 n=19+19)
FlowSetup/vectorize=true/distribute=false-24        208 ± 1%       197 ± 3%  -5.63%  (p=0.000 n=19+19)
FlowSetup/vectorize=false/distribute=true-24        206 ± 0%       197 ± 0%  -4.40%  (p=0.000 n=16+16)
FlowSetup/vectorize=false/distribute=false-24       197 ± 0%       188 ± 0%  -4.54%  (p=0.000 n=16+16)
```

Release justification: low-risk cleanup.

Release note: None

86956: roachtest: fix unintended exit when --cockroach-short is passed r=srosenberg a=srosenberg

Fixes a regression introduced by
#86625.

Release justification: fix roachtest nightly (on master)
Release note: None

Co-authored-by: Michael Erickson <michae2@cockroachlabs.com>
Co-authored-by: healthy-pod <ahmad@cockroachlabs.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: Stan Rosenberg <stan@cockroachlabs.com>
craig bot pushed a commit that referenced this issue Aug 28, 2022
@michae2
Merge #86932
e71c128 
86932: sql/stats: add cluster setting to disable generation of stats forecasts r=rytaft,msirek a=michae2

Add a new cluster setting `sql.stats.forecasts.enabled` that can be
used to disable generation of statistics forecasts in the stats cache.
Changing this setting will cause the stats cache to gradually evict and
reload all cached statistics as they are requested.

Fixes: #86350

Release justification: Low-risk update to new functionality.

Release note (sql change): Add a new cluster setting
`sql.stats.forecasts.enabled` which controls whether statistics
forecasts are generated by default for all tables. (Note that this is
different from the session setting `optimizer_use_forecasts` which
controls whether statistics forecasts are used when optimizing the
current query. If forecasting is disabled, then even if
`optimizer_use_forecasts` is true for a given query it won't have any
forecasts to use.)

Co-authored-by: Michael Erickson <michae2@cockroachlabs.com>
@craig craig bot closed this as completed in 896cf3c Aug 28, 2022
SQL Queries automation moved this from Active to Done Aug 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@michae2 michae2

Labels
A-sql-optimizer SQL logical planning and optimizations. A-sql-table-stats Table statistics (and their automatic refresh). C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team
Projects
SQL Queries
Archived in project
SQL Queries
Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sql/stats: add cluster setting to disable generation of stats forecasts michae2/cockroach
1 participant
@michae2