New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
*: replace loader by lightning #2243
Changes from 5 commits
e0e55b2
a36a132
e1189c3
a1a0e93
f8e20e8
b8ef039
bff9b7e
c7d38b5
54bea92
44c4055
78fe6cc
e0552a2
aa00b80
9519463
967173e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,95 @@ | ||||||
--- | ||||||
title: 使用 mydumper/tidb lightning 进行备份与恢复 | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
category: how-to | ||||||
aliases: ['/docs-cn/dev/how-to/maintain/backup-and-restore/mydumper-lightning'] | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
--- | ||||||
|
||||||
# 备份与恢复 | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
本文档将详细介绍如何使用 `mydumper`/`tidb lightning` 对 TiDB 进行全量备份与恢复。增量备份与恢复可使用 [TiDB Binlog](/dev/reference/tidb-binlog/overview.md)。 | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Either There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. +1 |
||||||
|
||||||
这里我们假定 TiDB 服务信息如下: | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 在技术文档中,应避免使用第一人称“我”、“我们”。 |
||||||
|
||||||
|Name|Address|Port|User|Password| | ||||||
|----|-------|----|----|--------| | ||||||
|TiDB|127.0.0.1|4000|root|*| | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yep |
||||||
|
||||||
在这个备份恢复过程中,我们会用到下面的工具: | ||||||
|
||||||
- [Mydumper](/dev/reference/tools/mydumper.md) 从 TiDB 导出数据 | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
- [TiDB Lightning](/dev/reference/tools/tidb-lightning/overview.md) 导入数据到 TiDB | ||||||
|
||||||
## 使用 `mydumper`/`tidb lightning` 全量备份恢复数据 | ||||||
|
||||||
`mydumper` 是一个强大的数据备份工具,具体可以参考 [`maxbube/mydumper`](https://github.com/maxbube/mydumper)。 | ||||||
|
||||||
可使用 [`mydumper`](/dev/reference/tools/mydumper.md) 从 TiDB 导出数据进行备份,然后用 [TiDB Lightning](/dev/reference/tools/tidb-lightning/overview.md) 将其导入到 TiDB 里面进行恢复。 | ||||||
|
||||||
> **注意:** | ||||||
> | ||||||
> PingCAP 研发团队对 `mydumper` 进行了针对 TiDB 的适配性改造,建议使用 PingCAP 官方提供的 [`mydumper`](/dev/reference/tools/mydumper.md)。由于使用 `mysqldump` 进行数据备份和恢复都要耗费许多时间,这里也并不推荐。 | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
### `mydumper`/`tidb lightning` 全量备份恢复最佳实践 | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
为了快速地备份恢复数据 (特别是数据量巨大的库),可以参考以下建议: | ||||||
|
||||||
* 使用 `mydumper` 导出来的数据文件尽可能的小, 可以通过设置参数 `-F` 来控制导出来的文件大小。如果后续使用 TiDB Lightning 对备份文件进行恢复,建议把 `mydumper` -F 参数的值设置为 `256`(单位 MB);如果使用 `loader` 恢复,则建议设置为 `64`(单位 MB)。 | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
## 从 TiDB 备份数据 | ||||||
|
||||||
我们使用 `mydumper` 从 TiDB 备份数据,如下: | ||||||
|
||||||
{{< copyable "shell-regular" >}} | ||||||
|
||||||
```bash | ||||||
./bin/mydumper -h 127.0.0.1 -P 4000 -u root -t 32 -F 256 -B test -T t1,t2 --skip-tz-utc -o ./var/test | ||||||
``` | ||||||
|
||||||
上面,我们使用 `-B test` 表明是对 `test` 这个 database 操作,然后用 `-T t1,t2` 表明只导出 `t1`,`t2` 两张表。 | ||||||
|
||||||
`-t 32` 表明使用 32 个线程去导出数据。`-F 256` 是将实际的 table 切分成多大的 chunk,这里就是 256MB 一个 chunk。 | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||
|
||||||
`--skip-tz-utc` 添加这个参数忽略掉 TiDB 与导数据的机器之间时区设置不一致的情况,禁止自动转换。 | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 原句几乎读不通。 |
||||||
|
||||||
如果 `mydumper` 出现以下报错: | ||||||
|
||||||
``` | ||||||
** (mydumper:27528): CRITICAL **: 13:25:09.081: Could not read data from testSchema.testTable: GC life time is shorter than transaction duration, transaction starts at 2019-08-05 21:10:01.451 +0800 CST, GC safe point is 2019-08-05 21:14:53.801 +0800 CST | ||||||
``` | ||||||
|
||||||
就再执行两步命令: | ||||||
|
||||||
1. 执行 `mydumper` 命令前,查询 TiDB 集群的 GC 值并使用 MySQL 客户端将其调整为合适的值。 | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 读者可能不知道什么是 GC,这里最好能写全 |
||||||
|
||||||
{{< copyable "sql" >}} | ||||||
|
||||||
```sql | ||||||
SELECT * FROM mysql.tidb WHERE VARIABLE_NAME = 'tikv_gc_life_time'; | ||||||
``` | ||||||
|
||||||
``` | ||||||
+-----------------------+------------------------------------------------------------------------------------------------+ | ||||||
| VARIABLE_NAME | VARIABLE_VALUE | | ||||||
+-----------------------+------------------------------------------------------------------------------------------------+ | ||||||
| tikv_gc_life_time | 10m0s | | ||||||
+-----------------------+------------------------------------------------------------------------------------------------+ | ||||||
1 rows in set (0.02 sec) | ||||||
``` | ||||||
|
||||||
{{< copyable "sql" >}} | ||||||
|
||||||
```sql | ||||||
update mysql.tidb set VARIABLE_VALUE = '720h' where VARIABLE_NAME = 'tikv_gc_life_time'; | ||||||
``` | ||||||
|
||||||
2. 执行 `mydumper` 命令后,将 TiDB 集群的 GC 值恢复到第 1 步中的初始值。 | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
{{< copyable "sql" >}} | ||||||
|
||||||
```sql | ||||||
update mysql.tidb set VARIABLE_VALUE = '10m' where VARIABLE_NAME = 'tikv_gc_life_time'; | ||||||
``` | ||||||
|
||||||
## 向 TiDB 恢复数据 | ||||||
|
||||||
我们使用 `tidb lightning` 将之前导出的数据导入到 TiDB,完成恢复操作。具体的使用方法见 [tidb lightning 使用文档](/dev/reference/tools/tidb-lightning/tidb-backend.md) | ||||||
IANTHEREAL marked this conversation as resolved.
Show resolved
Hide resolved
|
This file was deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
如果 lightning 完全可以代替 loader 的方式,loader 的文档可以去掉,否则建议保留 2 种
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
不会移除 loader,只是不在官方备份恢复推荐使用方案中。