Skip to content

Commit

Permalink
port changes to 24.1
Browse files Browse the repository at this point in the history
  • Loading branch information
taroface committed Mar 28, 2024
1 parent 8e2692d commit c4ec112
Show file tree
Hide file tree
Showing 8 changed files with 73 additions and 39 deletions.
11 changes: 4 additions & 7 deletions src/current/_includes/v24.1/misc/force-index-selection.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,11 +135,8 @@ Time: 1ms total (execution 1ms / network 0ms)

##### Prevent full scan

To prevent the optimizer from planning a full scan for a table, specify the `NO_FULL_SCAN` index hint. For example:
{% include {{ page.version.version }}/sql/no-full-scan.md %}

~~~sql
SELECT * FROM table_name@{NO_FULL_SCAN};
~~~

To prevent a full scan of a [partial index](#force-partial-index-scan), you must specify `NO_FULL_SCAN` _in combination with_ the partial index using `FORCE_INDEX=index_name`.
If you specify only `NO_FULL_SCAN`, a full scan of a partial index may be planned.
{{site.data.alerts.callout_success}}
For other methods of preventing full scans, see [Prevent the optimizer from planning full scans]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#prevent-the-optimizer-from-planning-full-scans).
{{site.data.alerts.end}}
3 changes: 1 addition & 2 deletions src/current/_includes/v24.1/misc/session-vars.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@
| <a id="default-transaction-quality-of-service"></a> `default_transaction_quality_of_service` | The default transaction quality of service for the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level]({% link {{ page.version.version }}/admission-control.md %}#set-quality-of-service-level-for-a-session). | `regular` | Yes | Yes |
| <a id="default-transaction-read-only"></a> `default_transaction_read_only` | The default transaction access mode for the current session. <br/>If set to `on`, only read operations are allowed in transactions in the current session; if set to `off`, both read and write operations are allowed. See [`SET TRANSACTION`]({% link {{ page.version.version }}/set-transaction.md %}) for more details. | `off` | Yes | Yes |
| <a id="default-transaction-use-follower-reads"></a> `default_transaction_use_follower_reads` | If set to on, all read-only transactions use [`AS OF SYSTEM TIME follower_read_timestamp()`]({% link {{ page.version.version }}/as-of-system-time.md %}) to allow the transaction to use follower reads. <br/>If set to `off`, read-only transactions will only use follower reads if an `AS OF SYSTEM TIME` clause is specified in the statement, with an interval of at least 4.8 seconds. | `off` | Yes | Yes |
| <a id="disallow-full-table-scans"></a> `disallow_full_table_scans` | If set to `on`, all queries that have planned a full table or full secondary index scan will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes |
| <a id="distsql"></a> `distsql` | The query distribution mode for the session. By default, CockroachDB determines which queries are faster to execute if distributed across multiple nodes, and all other queries are run through the gateway node. | `auto` | Yes | Yes |
| <a id="disallow-full-table-scans"></a> `disallow_full_table_scans` | If set to `on`, queries on "large" tables with a row count greater than [`large_full_scan_rows`](#large-full-scan-rows) will not use full table or index scans. If no other query plan is possible, queries will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes || <a id="distsql"></a> `distsql` | The query distribution mode for the session. By default, CockroachDB determines which queries are faster to execute if distributed across multiple nodes, and all other queries are run through the gateway node. | `auto` | Yes | Yes |
| <a id="enable-auto-rehoming"></a> `enable_auto_rehoming` | When enabled, the [home regions]({% link {{ page.version.version }}/alter-table.md %}#crdb_region) of rows in [`REGIONAL BY ROW`]({% link {{ page.version.version }}/alter-table.md %}#set-the-table-locality-to-regional-by-row) tables are automatically set to the region of the [gateway node]({% link {{ page.version.version }}/ui-sessions-page.md %}#session-details-gateway-node) from which any [`UPDATE`]({% link {{ page.version.version }}/update.md %}) or [`UPSERT`]({% link {{ page.version.version }}/upsert.md %}) statements that operate on those rows originate. | `off` | Yes | Yes |
| <a id="enable-durable-locking-for-serializable"></a> `enable_durable_locking_for_serializable` | Indicates whether CockroachDB replicates [`FOR UPDATE` and `FOR SHARE`]({% link {{ page.version.version }}/select-for-update.md %}#lock-strengths) locks via [Raft]({% link {{ page.version.version }}/architecture/replication-layer.md %}#raft), allowing locks to be preserved when leases are transferred. Note that replicating `FOR UPDATE` and `FOR SHARE` locks will add latency to those statements. This setting only affects `SERIALIZABLE` transactions and matches the default `READ COMMITTED` behavior when enabled. | `off` | Yes | Yes |
| <a id="enable-implicit-fk-locking-for-serializable"></a> `enable_implicit_fk_locking_for_serializable` | Indicates whether CockroachDB uses [shared locks]({% link {{ page.version.version }}/select-for-update.md %}#lock-strengths) to perform [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) checks. To take effect, `enable_shared_locking_for_serializable` must also be enabled. This setting only affects `SERIALIZABLE` transactions and matches the default `READ COMMITTED` behavior when enabled. | `off` | Yes | Yes |
Expand Down
12 changes: 12 additions & 0 deletions src/current/_includes/v24.1/sql/disallow-full-table-scans.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
- At the cluster level, set `disallow_full_table_scans` for some or all users/roles. For example:

{% include_cached copy-clipboard.html %}
~~~ sql
ALTER ROLE ALL SET disallow_full_table_scans = true;
~~~

- At the application level, add `disallow_full_table_scans` to the connection string using the [`options` parameter]({% link {{page.version.version}}/connection-parameters.md %}#additional-connection-parameters).

If you disable full scans, you can set the [`large_full_scan_rows` session variable]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows) to specify the maximum table size allowed for a full scan. If no alternative plan is possible, the optimizer will return an error.

If you disable full scans, and you provide an [index hint]({% link {{ page.version.version }}/indexes.md %}#selection), the optimizer will try to avoid a full scan while also respecting the index hint. If this is not possible, the optimizer will return an error. If you do not provide an index hint and it is not possible to avoid a full scan, the optimizer will return an error, the full scan will be logged, and the `sql.guardrails.full_scan_rejected.count` [metric]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) will be updated.
15 changes: 15 additions & 0 deletions src/current/_includes/v24.1/sql/no-full-scan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
- To prevent the optimizer from planning a full scan for a specific table, specify the `NO_FULL_SCAN` index hint. For example:

{% include_cached copy-clipboard.html %}
~~~ sql
SELECT * FROM table_name@{NO_FULL_SCAN};
~~~

- To prevent a full scan of a [partial index]({% link {{ page.version.version }}/partial-indexes.md %}) for a specific table, you must specify `NO_FULL_SCAN` in combination with the index name using [`FORCE_INDEX`]({% link {{ page.version.version }}/table-expressions.md %}#force-index-selection). For example:

{% include_cached copy-clipboard.html %}
~~~ sql
SELECT * FROM table_name@{FORCE_INDEX=index_name,NO_FULL_SCAN} WHERE b > 0;
~~~

This will force a constrained scan of the partial index. If a constrained scan of the partial index is not possible, an error will be returned.
15 changes: 15 additions & 0 deletions src/current/_includes/v24.1/sql/transactions-limit-rows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
- When the `transaction_rows_read_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-err) is enabled, transactions that read more than the specified number of rows will fail. In addition, the [optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) will not create query plans with scans that exceed the specified row limit. For example, to set a default value for all users at the cluster level:

{% include_cached copy-clipboard.html %}
~~~ sql
ALTER ROLE ALL SET transaction_rows_read_err = 1000;
~~~

- When the `transaction_rows_written_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-written-err) is enabled, transactions that write more than the specified number of rows will fail. For example, to set a default value for all users at the cluster level:

{% include_cached copy-clipboard.html %}
~~~ sql
ALTER ROLE ALL SET transaction_rows_written_err = 1000;
~~~

To assess the impact of enabling these settings, use the corresponding session settings [`transaction_rows_read_log`]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-log) and [`transaction_rows_written_log`]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-written-log) to log transactions that read or write the specified number of rows. Transactions are logged to the [`SQL_PERF`]({% link {{ page.version.version }}/logging.md %}#sql_perf) channel.
4 changes: 1 addition & 3 deletions src/current/v24.1/cost-based-optimizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,9 +232,7 @@ Even if you have [secondary indexes]({% link {{ page.version.version }}/schema-d

You can disable statement plans that perform full table scans with the [`disallow_full_table_scans` session variable]({% link {{ page.version.version }}/set-vars.md %}#disallow-full-table-scans).

If you disable full scans, you can set the [`large_full_scan_rows` session variable]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows) to specify the maximum table size allowed for a full scan. If no alternative plan is possible, the optimizer will return an error.

If you disable full scans, and you provide an [index hint]({% link {{ page.version.version }}/indexes.md %}#selection), the optimizer will try to avoid a full scan while also respecting the index hint. If this is not possible, the optimizer will return an error. If you do not provide an index hint, the optimizer will return an error, the full scan will be logged, and the `sql.guardrails.full_scan_rejected.count` [metric]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) will be updated.
{% include {{ page.version.version }}/sql/disallow-full-table-scans.md %}

## Control whether the optimizer uses an index

Expand Down
33 changes: 22 additions & 11 deletions src/current/v24.1/performance-best-practices-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,9 +263,7 @@ See [Authorization Best Practices]({% link {{ page.version.version }}/security-r

For large tables, avoid table scans (that is, reading the entire table data) whenever possible. Instead, define the required fields in a `SELECT` statement.

#### Example

Suppose the table schema is as follows:
For example, suppose the table schema is as follows:

{% include_cached copy-clipboard.html %}
~~~ sql
Expand Down Expand Up @@ -298,24 +296,37 @@ This query returns the account balances of the customers.

`SELECT DISTINCT` allows you to obtain unique entries from a query by removing duplicate entries. However, `SELECT DISTINCT` is computationally expensive. As a performance best practice, use [`SELECT` with the `WHERE` clause]({% link {{ page.version.version }}/select-clause.md %}#filter-rows) instead.

### Use secondary indexes to optimize queries

See [Statement Tuning with `EXPLAIN`]({% link {{ page.version.version }}/sql-tuning-with-explain.md %}#issue-full-table-scans).

### Use `AS OF SYSTEM TIME` to decrease conflicts with long-running queries

If you have long-running queries (such as analytics queries that perform full table scans) that can tolerate slightly out-of-date reads, consider using the [`... AS OF SYSTEM TIME` clause]({% link {{ page.version.version }}/select-clause.md %}#select-historical-data-time-travel). Using this, your query returns data as it appeared at a distinct point in the past and will not cause [conflicts]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#transaction-conflicts) with other concurrent transactions, which can increase your application's performance.

However, because `AS OF SYSTEM TIME` returns historical data, your reads might be stale.

### Disallow full table scans with the `disallow_full_table_scans` setting
### Prevent the optimizer from planning full scans

To avoid overloading production clusters, there are several ways to prevent the [cost-based-optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) from generating query plans with [full table and index scans]({% link {{ page.version.version }}/ui-sql-dashboard.md %}#full-table-index-scans).

#### Use index hints to prevent full scans on tables

{% include {{ page.version.version }}/sql/no-full-scan.md %}

#### Disallow query plans that use full scans

When the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) is enabled, the optimizer will not plan full table or index scans on "large" tables (i.e., those with more rows than [`large_full_scan_rows`]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows)).

{% include {{ page.version.version }}/sql/disallow-full-table-scans.md %}

To prevent overloading production clusters with [full table scans]({% link {{ page.version.version }}/ui-sql-dashboard.md %}#full-table-index-scans), you have several options:
#### Disallow query plans that scan more than a number of rows

1. At the cluster level, configure the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) for some or all users/roles using the [`ALTER ROLE`]({% link {{ page.version.version }}/alter-role.md %}) statement.
When the `transaction_rows_read_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-err) is enabled, the [optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) will not create query plans with scans that exceed the specified row limit. See [Disallow transactions from reading or writing many rows](#disallow-transactions-from-reading-or-writing-many-rows).

{% include_cached copy-clipboard.html %}
~~~ sql
ALTER ROLE ALL SET disallow_full_table_scans = true;
~~~
### Disallow transactions from reading or writing many rows

1. At the application level, add the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) to the connection string using the [`options` parameter]({% link {{page.version.version}}/connection-parameters.md %}#additional-connection-parameters).
{% include {{ page.version.version }}/sql/transactions-limit-rows.md %}

<a id="understanding-and-avoiding-transaction-contention"></a>

Expand Down
19 changes: 3 additions & 16 deletions src/current/v24.1/transactions.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,27 +228,14 @@ For more information about the relationship between these levels, see [A Critiqu

You can limit the number of rows written or read in a transaction at the cluster or session level. This allows you configure CockroachDB to log or reject statements that could destabilize a cluster or violate application best practices.

Use the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}) `sql.defaults.transaction_rows_written_log`,
`sql.defaults.transaction_rows_written_err`, `sql.defaults.transaction_rows_read_log`, and
`sql.defaults.transaction_rows_read_err` and [session settings]({% link {{ page.version.version }}/set-vars.md %}) `transaction_rows_written_log`,
`transaction_rows_written_err`, `transaction_rows_read_log`, and
`transaction_rows_read_err` to limit the number of rows written or read in a
transaction. When the `log` limit is reached, the transaction is logged to the `SQL_PERF` channel.
When the `err` limit is reached, the transaction is rejected. The limits are enforced after each
statement of a transaction has been fully executed.

The "write" limits apply to `INSERT`, `INSERT INTO SELECT FROM`, `INSERT ON CONFLICT`, `UPSERT`, `UPDATE`,
and `DELETE` SQL statements. The "read" limits apply to the `SELECT`
statement in addition to the statements subject to the "write" limits. The limits **do not**
apply to `CREATE TABLE AS`, `SELECT`, `IMPORT`, `TRUNCATE`, `DROP`, `ALTER TABLE`, `BACKUP`,
`RESTORE`, or `CREATE STATISTICS` statements.
{% include {{ page.version.version }}/sql/transactions-limit-rows.md %}

The limits are enforced after each statement of a transaction has been fully executed. The "write" limits apply to `INSERT`, `INSERT INTO SELECT FROM`, `INSERT ON CONFLICT`, `UPSERT`, `UPDATE`, and `DELETE` SQL statements. The "read" limits apply to the `SELECT` statement in addition to the statements subject to the "write" limits. The limits **do not** apply to `CREATE TABLE AS`, `IMPORT`, `TRUNCATE`, `DROP`, `ALTER TABLE`, `BACKUP`, `RESTORE`, or `CREATE STATISTICS` statements.

{{site.data.alerts.callout_info}}
Enabling `transaction_rows_read_err` disables a performance optimization for mutation statements in implicit transactions where CockroachDB can auto-commit without additional network round trips.
{{site.data.alerts.end}}

{% include {{page.version.version}}/sql/sql-defaults-cluster-settings-deprecation-notice.md %}

## See also

- [`BEGIN`]({% link {{ page.version.version }}/begin-transaction.md %})
Expand Down

0 comments on commit c4ec112

Please sign in to comment.