TiFlash results materialization #11701
Conversation
|
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
ti-chi-bot
added
the
do-not-merge/work-in-progress
ti-chi-bot
added
missing-translation-status
moyun
changed the title
qiancai
added
translation/doing
ti-chi-bot
removed
the
missing-translation-status
moyun
changed the title
ti-chi-bot
removed
the
do-not-merge/work-in-progress
|
|
||
| ### 限制 | ||
|
|
||
| * 对写入部分事务大小（SELECT 子句返回的结果集）的硬性限制为 1 GB，推荐的使用场景是 100 MB以下。 |
There was a problem hiding this comment.
这里可以引用一下 https://docs.pingcap.com/zh/tidb/dev/transaction-overview#%E4%BA%8B%E5%8A%A1%E9%99%90%E5%88%B6 ？
这里的逻辑跟普通写入事务是一样的。默认事务大小是 100MB，要更大得调整 txn-total-size-limit.
There was a problem hiding this comment.
事务太大还有另外一种方法是 https://docs.pingcap.com/zh/tidb/dev/system-variables#tidb_enable_batch_dml
把大事务拆成多个小事务，我理解这个也是值得一提的？如果出错用户可以删表重新来。
| * TiDB 对 `INSERT INTO SELECT` 语句的并发没有硬性限制，但是推荐考虑以下用法： | ||
|
|
||
| * 当“写事务”较大时，例如接近 1 GB， 建议控制并发不超过 10。 | ||
| * 当“写事务”较小时，例如小于 100 MB， 建议控制并发不超过 30。 |
There was a problem hiding this comment.
限制这个还是没有改，根据事务文档 https://docs.pingcap.com/zh/tidb/stable/transaction-overview#%E4%BA%8B%E5%8A%A1%E9%99%90%E5%88%B6。
TiDB 中，单个事务的总大小默认不超过 100 MB，这个默认值可以通过配置文件中的配置项 txn-total-size-limit 进行修改，最大支持 1 TB。单个事务的实际大小限制还取决于服务器剩余可用内存的大小，执行事务时 TiDB 进程的内存消耗相对于事务大小会存在一定程度的放大，最大可能达到提交事务大小的 2 到 3 倍以上。
There was a problem hiding this comment.
TiDB 对 SELECT 子句返回的结果集（即 INSERT 写入的事务）大小的硬性限制为 1 GB，推荐的使用场景是 100 MB 以下。
若 SELECT 返回结果大小超过了 1 GB，那么整条语句将会被强制终止。用户会得到以下出错信息:
The query produced a too large intermediate result and thus failed
我建议这段直接引用事务的文档，因为这里逻辑是一样的，没有专门针对 insert select 的限制。
There was a problem hiding this comment.
限制这个还是没有改，根据事务文档 https://docs.pingcap.com/zh/tidb/stable/transaction-overview#%E4%BA%8B%E5%8A%A1%E9%99%90%E5%88%B6。
TiDB 中，单个事务的总大小默认不超过 100 MB，这个默认值可以通过配置文件中的配置项 txn-total-size-limit 进行修改，最大支持 1 TB。单个事务的实际大小限制还取决于服务器剩余可用内存的大小，执行事务时 TiDB 进程的内存消耗相对于事务大小会存在一定程度的放大，最大可能达到提交事务大小的 2 到 3 倍以上。
there is no conflict here with the tidb doc, and the conclusion here is specifically for this scenario but just happenly being same with the tidb doc.
Co-authored-by: Aolin <aolinz@outlook.com>
Co-authored-by: Aolin <aolinz@outlook.com>
update as suggested Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
Co-authored-by: Liqi Geng <gengliqiii@gmail.com>
Co-authored-by: Aolin <aolinz@outlook.com>
ti-chi-bot
added
status/LGT2
|
/merge |
|
This pull request has been accepted and is ready to merge. Commit hash: c90460f
|
ti-chi-bot
added
the
status/can-merge
First-time contributors' checklist
What is changed, added or deleted? (Required)
Add a new doc for explaining how TiFlash can save the query results to a TiDB table via a txn.
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions (in Chinese).
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?