consolidate guidance on avoiding full scans; update session vars #18338

taroface · 2024-02-29T23:15:37Z

DOC-9629

Added a section to SQL Performance Best Practices that summarizes how to prevent the optimizer from planning full scans. Preview
This also consolidates existing guidance across the docs into several includes.
Updated descriptions of the disallow_full_table_scans and large_full_scan_rows settings (for the past 3 docs versions) as it looked like the behavior of these settings was updated in sql, opt: support hint to disallow full scan, update coster to avoid full scans cockroach#71317 but this wasn't documented.

github-actions · 2024-02-29T23:15:57Z

Files changed:

src/current/_includes/v22.2/misc/session-vars.md:

src/current/_includes/v23.1/misc/session-vars.md:

src/current/_includes/v23.2/misc/force-index-selection.md:

src/current/_includes/v23.2/misc/session-vars.md:

src/current/_includes/v23.2/sql/disallow-full-table-scans.md:

src/current/_includes/v23.2/sql/no-full-scan.md:

src/current/v23.2/performance-best-practices-overview.md
src/current/_includes/v23.2/misc/force-index-selection.md:

src/current/_includes/v23.2/sql/transactions-limit-rows.md:

src/current/_includes/v24.1/misc/force-index-selection.md:

src/current/_includes/v24.1/misc/session-vars.md:

src/current/_includes/v24.1/sql/disallow-full-table-scans.md:

src/current/_includes/v24.1/sql/no-full-scan.md:

src/current/v24.1/performance-best-practices-overview.md
src/current/_includes/v24.1/misc/force-index-selection.md:

src/current/_includes/v24.1/sql/transactions-limit-rows.md:

src/current/v23.2/cost-based-optimizer.md
src/current/v23.2/performance-best-practices-overview.md
src/current/v23.2/transactions.md
src/current/v24.1/cost-based-optimizer.md
src/current/v24.1/performance-best-practices-overview.md
src/current/v24.1/transactions.md

netlify · 2024-02-29T23:16:18Z

✅ Deploy Preview for cockroachdb-api-docs canceled.

Name	Link
🔨 Latest commit	`b4eaf59`
🔍 Latest deploy log	https://app.netlify.com/sites/cockroachdb-api-docs/deploys/66071aff8c0e9500089da17e

netlify · 2024-02-29T23:16:22Z

✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.

Name	Link
🔨 Latest commit	`b4eaf59`
🔍 Latest deploy log	https://app.netlify.com/sites/cockroachdb-interactivetutorials-docs/deploys/66071affb172b50008f4df9c

netlify · 2024-02-29T23:20:06Z

✅ Netlify Preview

Name	Link
🔨 Latest commit	`b4eaf59`
🔍 Latest deploy log	https://app.netlify.com/sites/cockroachdb-docs/deploys/66071aff04a490000936fa7d
😎 Deploy Preview	https://deploy-preview-18338--cockroachdb-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

rytaft

This looks great, thanks! Sorry for the delay.

There was another support issue where the customer was tuning their queries and was concerned that the EXPLAIN output showed some full table scans. However, this was expected since the customer was testing with empty tables, and therefore the stats did not represent the production environment.

Do you think it makes sense to update this section of the docs to mention that most of the time the optimizer makes the right decision, and if you see a full scan you should (a) make sure that it's actually causing a performance problem, and (b) confirm that the optimizer would actually choose a full scan in the production environment (with correct stats, cluster topology, etc)? Only then should a customer try to use one of these tools to override the optimizer's decision.

If you think that should be a different PR, I'm happy to file a separate docs issue. Let me know what you think. Thanks!

src/current/_includes/v23.2/sql/disallow-full-table-scans.md

src/current/_includes/v23.2/sql/no-full-scan.md

src/current/v23.2/transactions.md

src/current/_includes/v23.2/sql/transactions-limit-rows.md

taroface · 2024-03-26T19:22:51Z

This looks great, thanks! Sorry for the delay.

There was another support issue where the customer was tuning their queries and was concerned that the EXPLAIN output showed some full table scans. However, this was expected since the customer was testing with empty tables, and therefore the stats did not represent the production environment.

Do you think it makes sense to update this section of the docs to mention that most of the time the optimizer makes the right decision, and if you see a full scan you should (a) make sure that it's actually causing a performance problem, and (b) confirm that the optimizer would actually choose a full scan in the production environment (with correct stats, cluster topology, etc)? Only then should a customer try to use one of these tools to override the optimizer's decision.

If you think that should be a different PR, I'm happy to file a separate docs issue. Let me know what you think. Thanks!

TFTR @rytaft! I had a follow-up question (above).

I do think that a separate docs issue with some more details would make sense for the other recommendations you describe.

rytaft · 2024-03-28T19:25:06Z

This looks great, thanks! Sorry for the delay.
There was another support issue where the customer was tuning their queries and was concerned that the EXPLAIN output showed some full table scans. However, this was expected since the customer was testing with empty tables, and therefore the stats did not represent the production environment.
Do you think it makes sense to update this section of the docs to mention that most of the time the optimizer makes the right decision, and if you see a full scan you should (a) make sure that it's actually causing a performance problem, and (b) confirm that the optimizer would actually choose a full scan in the production environment (with correct stats, cluster topology, etc)? Only then should a customer try to use one of these tools to override the optimizer's decision.
If you think that should be a different PR, I'm happy to file a separate docs issue. Let me know what you think. Thanks!

TFTR @rytaft! I had a follow-up question (above).

I do think that a separate docs issue with some more details would make sense for the other recommendations you describe.

Done: #18428

rytaft

LGTM thank you! And sorry for the delays!

mdlinville

Good refactor. I'll go ahead and approve because I don't feel super strongly about any of my feedback.

src/current/_includes/v22.2/misc/session-vars.md

src/current/_includes/v23.2/misc/force-index-selection.md

src/current/_includes/v23.2/misc/session-vars.md

src/current/_includes/v24.1/misc/session-vars.md

src/current/_includes/v24.1/sql/disallow-full-table-scans.md

src/current/_includes/v24.1/sql/transactions-limit-rows.md

src/current/v23.2/transactions.md

taroface requested a review from rytaft February 29, 2024 23:15

exalate-issue-sync bot mentioned this pull request Mar 4, 2024

Consider adding section on guardrails to avoid full scan #18279

Closed

rytaft approved these changes Mar 26, 2024

View reviewed changes

rytaft mentioned this pull request Mar 28, 2024

add guidance on using correct stats when tuning queries with EXPLAIN #18428

Open

taroface added 2 commits March 28, 2024 16:32

consolidate guidance on avoiding full scans; update session vars

7c8353e

address rytaft feedback

8e2692d

rytaft approved these changes Mar 28, 2024

View reviewed changes

port changes to 24.1

c4ec112

taroface force-pushed the full-scan-guardrails branch from b7c44db to c4ec112 Compare March 28, 2024 20:40

taroface requested a review from mdlinville March 28, 2024 20:40

mdlinville approved these changes Mar 29, 2024

View reviewed changes

taroface force-pushed the full-scan-guardrails branch 2 times, most recently from 0f27f7e to 16e9a05 Compare March 29, 2024 19:37

Apply suggestions from code review

b4eaf59

taroface force-pushed the full-scan-guardrails branch from 16e9a05 to b4eaf59 Compare March 29, 2024 19:48

taroface merged commit 6ee9ceb into main Mar 29, 2024
6 checks passed

taroface deleted the full-scan-guardrails branch March 29, 2024 19:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

consolidate guidance on avoiding full scans; update session vars #18338

consolidate guidance on avoiding full scans; update session vars #18338

taroface commented Feb 29, 2024 •

edited

github-actions bot commented Feb 29, 2024 •

edited

netlify bot commented Feb 29, 2024 •

edited

netlify bot commented Feb 29, 2024 •

edited

netlify bot commented Feb 29, 2024 •

edited

rytaft left a comment

taroface commented Mar 26, 2024

rytaft commented Mar 28, 2024

rytaft left a comment

mdlinville left a comment

consolidate guidance on avoiding full scans; update session vars #18338

consolidate guidance on avoiding full scans; update session vars #18338

Conversation

taroface commented Feb 29, 2024 • edited

github-actions bot commented Feb 29, 2024 • edited

Files changed:

netlify bot commented Feb 29, 2024 • edited

✅ Deploy Preview for cockroachdb-api-docs canceled.

netlify bot commented Feb 29, 2024 • edited

✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.

netlify bot commented Feb 29, 2024 • edited

✅ Netlify Preview

rytaft left a comment

Choose a reason for hiding this comment

taroface commented Mar 26, 2024

rytaft commented Mar 28, 2024

rytaft left a comment

Choose a reason for hiding this comment

mdlinville left a comment

Choose a reason for hiding this comment

taroface commented Feb 29, 2024 •

edited

github-actions bot commented Feb 29, 2024 •

edited

netlify bot commented Feb 29, 2024 •

edited

netlify bot commented Feb 29, 2024 •

edited

netlify bot commented Feb 29, 2024 •

edited