tidb-configuration-file: add more notes to txn-entry-size-limit

[dveeden]

What is changed, added or deleted? (Required)

Growing the txn-entry-size-limit to the maximum size needs adjustments in other settings as well.

When txn-entry-size-limit is too small:

sql> INSERT INTO t1(t) VALUES(REPEAT('x',6*1024*1024));
ERROR: 8025 (HY000): entry too large, the max entry size is 6291456, the size of data is 6291489

When raft-entry-max-size is too small:

sql> INSERT INTO t1(t) VALUES(REPEAT('x',15*1024*1024));
ERROR: 1105 (HY000): message:"raft entry is too large, region 2, entry size 15728790" raft_entry_too_large:<region_id:2 entry_size:15728790 > 

When max_allowed_packet is too small:

sql> INSERT INTO t1(t) VALUES(REPEAT('x',100*1024*1024));
ERROR: 1301 (HY000): Result of repeat() was larger than max_allowed_packet (67108864) - truncated

When txn-total-size-limit is too small:

sql> INSERT INTO t1(t) VALUES(REPEAT('x',100*1024*1024));
ERROR: 8004 (HY000): Transaction is too large, size: 104857633

Tested with:

tiup playground --tiflash 0 --without-monitor --db.config /tmp/tidb.toml --kv.config /tmp/tikv.toml v6.3.0

tikv.toml:

[raftstore]
raft-entry-max-size = "150MB"

tidb.toml:

[performance]
txn-entry-size-limit = 125829120
txn-total-size-limit = 1073741824

And SET GLOBAL max_allowed_packet=130*1024*1024 (and then reconnect)

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

• master (the latest development version)
• v6.4 (TiDB 6.4 versions)
• v6.3 (TiDB 6.3 versions)
• v6.1 (TiDB 6.1 versions)
• v5.4 (TiDB 5.4 versions)
• v5.3 (TiDB 5.3 versions)
• v5.2 (TiDB 5.2 versions)
• v5.1 (TiDB 5.1 versions)
• v5.0 (TiDB 5.0 versions)

What is the related PR or file link(s)?

• This PR is translated from:
• Other reference link(s):

Do your changes match any of the following descriptions?

• Delete files
• Change aliases
• Need modification after applied to another branch
• Might cause conflicts after applied to another branch

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• TomShawn
• qiancai

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[dveeden]

Note that the actual limit might be slightly less. Instead of 6MiB (default config) exactly it is 33 bytes less for the defaults. I don't think this is a problem.

sql> CREATE TABLE t1(id bigint primary key auto_random, t longtext);
Query OK, 0 rows affected, 1 warning (0.1575 sec)
Note (code 1105): Available implicit allocation times: 288230376151711743

sql> INSERT INTO t1(t) VALUES(REPEAT('x',5*1024*1024));
Query OK, 1 row affected (0.0811 sec)

sql> INSERT INTO t1(t) VALUES(REPEAT('x',6*1024*1024));
ERROR: 8025 (HY000): entry too large, the max entry size is 6291456, the size of data is 6291489

sql> SELECT 6291489-6291456;
+-----------------+
| 6291489-6291456 |
+-----------------+
|              33 |
+-----------------+
1 row in set (0.0003 sec)

sql> INSERT INTO t1(t) VALUES(REPEAT('x',(6*1024*1024)-33));
Query OK, 1 row affected (0.0803 sec)

sql> SELECT LENGTH(t), FORMAT_BYTES(LENGTH(t)) FROM t1;
+-----------+-------------------------+
| LENGTH(t) | FORMAT_BYTES(LENGTH(t)) |
+-----------+-------------------------+
|   6291423 | 6.00 MiB                |
|   5242880 | 5.00 MiB                |
+-----------+-------------------------+
2 rows in set (0.0921 sec)

[TomShawn]

What does "rows that are bigger" mean? Does it mean rows bigger than txn-entry-size-limit?

Suggested change

	- The default `max_allowed_packet` is 67108864 (64 MiB), which may lead to truncation with rows that are bigger.
	- The default `txn-total-size-limit` is 100 MiB, which you need to increase if you increase `txn-entry-size-limit` to be over 100 MiB.
	- Because the default value of [`max_allowed_packet`](/system-variables.md#max_allowed_packet) (the maximum size of a packet for the MySQL protocol) is 67108864 (64 MiB), which might lead to truncation with rows that are bigger.
	- The default value of [`txn-total-size-limit`](#txn-total-size-limit) (the size limit of a single transaction in TiDB) is 100 MiB. If you increase the `txn-entry-size-limit` value to be over 100 MiB, you need to increase the `txn-total-size-limit` value accordingly.

[dveeden]

If a column is larger than max_allowed_packet it gets truncated. With a strict SQL mode it will probably give an error instead of truncating the value, but I didn't test that.

[TomShawn]

rest LGTM

[dveeden]

@TomShawn PTAL

[dveeden]

@TomShawn PTAL

[qiancai]

In L361, do we need to mention this configuration item also controls the size limit of a single column?

[dveeden]

I don't think that's needed as this should be implicit as columns can't be bigger than the row.

[dveeden]

The ci / tidb-check found this:

 tidb-configuration-file.md
  365:24-365:87  warning  Dead anchor: /system-variables.md#max_allowed_packet  pingcap-docs-anchors  remark-lint

However the link seems fine to me.

[TomShawn]

/rebase

[TomShawn]

The ci / tidb-check found this:

 tidb-configuration-file.md
  365:24-365:87  warning  Dead anchor: /system-variables.md#max_allowed_packet  pingcap-docs-anchors  remark-lint

However the link seems fine to me.

Anchor fixed. If @qiancai has no further comment, let's merge this PR.

[qiancai]

LGTM

[qiancai]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 64542ef

[ti-chi-bot]

In response to a cherrypick label: new pull request created: #11014.

[ti-chi-bot]

In response to a cherrypick label: new pull request created: #11015.