Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: replace loader by lightning #2243

Merged
merged 15 commits into from Feb 15, 2020
2 changes: 1 addition & 1 deletion dev/TOC.md
Expand Up @@ -60,7 +60,7 @@
+ 运维
- [Ansible 常见运维操作](/dev/how-to/maintain/ansible-operations.md)
+ 备份与恢复
- [使用 Mydumper/Loader 进行备份与恢复](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果 lightning 完全可以代替 loader 的方式,loader 的文档可以去掉,否则建议保留 2 种

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不会移除 loader,只是不在官方备份恢复推荐使用方案中。

- [使用 Mydumper/Loader 进行备份与恢复](/dev/how-to/maintain/backup-and-restore/mydumper-lightning.md)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- [使用 Mydumper/Loader 进行备份与恢复](/dev/how-to/maintain/backup-and-restore/mydumper-lightning.md)
- [使用 Mydumper/Lightning 进行备份与恢复](/dev/how-to/maintain/backup-and-restore/mydumper-lightning.md)

- [使用 BR 进行备份与恢复](/dev/reference/tools/br/br.md)
- [BR 备份与恢复最佳实践](/dev/reference/tools/br/br-best-practices.md)
+ 定位异常查询
Expand Down
95 changes: 95 additions & 0 deletions dev/how-to/maintain/backup-and-restore/mydumper-lightning.md
@@ -0,0 +1,95 @@
---
title: 使用 mydumper/tidb lightning 进行备份与恢复
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved
category: how-to
aliases: ['/docs-cn/dev/how-to/maintain/backup-and-restore/mydumper-lightning']
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved
---

# 备份与恢复
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved

本文档将详细介绍如何使用 `mydumper`/`tidb lightning` 对 TiDB 进行全量备份与恢复。增量备份与恢复可使用 [TiDB Binlog](/dev/reference/tidb-binlog/overview.md)。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either tidb-lightning (the exe name) or "TiDB Lightning" (the product name)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1


这里我们假定 TiDB 服务信息如下:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
这里我们假定 TiDB 服务信息如下
这里假定 TiDB 的服务信息如下

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

在技术文档中,应避免使用第一人称“我”、“我们”。


|Name|Address|Port|User|Password|
|----|-------|----|----|--------|
|TiDB|127.0.0.1|4000|root|*|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

* = no password?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep


在这个备份恢复过程中,我们会用到下面的工具:

- [Mydumper](/dev/reference/tools/mydumper.md) 从 TiDB 导出数据
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved
- [TiDB Lightning](/dev/reference/tools/tidb-lightning/overview.md) 导入数据到 TiDB

## 使用 `mydumper`/`tidb lightning` 全量备份恢复数据

`mydumper` 是一个强大的数据备份工具,具体可以参考 [`maxbube/mydumper`](https://github.com/maxbube/mydumper)。

可使用 [`mydumper`](/dev/reference/tools/mydumper.md) 从 TiDB 导出数据进行备份,然后用 [TiDB Lightning](/dev/reference/tools/tidb-lightning/overview.md) 将其导入到 TiDB 里面进行恢复。

> **注意:**
>
> PingCAP 研发团队对 `mydumper` 进行了针对 TiDB 的适配性改造,建议使用 PingCAP 官方提供的 [`mydumper`](/dev/reference/tools/mydumper.md)。由于使用 `mysqldump` 进行数据备份和恢复都要耗费许多时间,这里也并不推荐。
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved

### `mydumper`/`tidb lightning` 全量备份恢复最佳实践
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved

为了快速地备份恢复数据 (特别是数据量巨大的库),可以参考以下建议:

* 使用 `mydumper` 导出来的数据文件尽可能的小, 可以通过设置参数 `-F` 来控制导出来的文件大小。如果后续使用 TiDB Lightning 对备份文件进行恢复,建议把 `mydumper` -F 参数的值设置为 `256`(单位 MB);如果使用 `loader` 恢复,则建议设置为 `64`(单位 MB)。
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved

## 从 TiDB 备份数据

我们使用 `mydumper` 从 TiDB 备份数据,如下:

{{< copyable "shell-regular" >}}

```bash
./bin/mydumper -h 127.0.0.1 -P 4000 -u root -t 32 -F 256 -B test -T t1,t2 --skip-tz-utc -o ./var/test
```

上面,我们使用 `-B test` 表明是对 `test` 这个 database 操作,然后用 `-T t1,t2` 表明只导出 `t1`,`t2` 两张表。

`-t 32` 表明使用 32 个线程去导出数据。`-F 256` 是将实际的 table 切分成多大的 chunk,这里就是 256MB 一个 chunk。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`-t 32` 表明使用 32 个线程去导出数据。`-F 256` 是将实际的 table 切分成多大的 chunk,这里就是 256MB 一个 chunk
`-t 32` 表示使用 32 个线程去导出数据。`-F 256` 是将实际的表切分成一定大小的 chunk,这里的 chunk 大小为 256MB

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

切分成多大的 不是正式用语。


`--skip-tz-utc` 添加这个参数忽略掉 TiDB 与导数据的机器之间时区设置不一致的情况,禁止自动转换。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`--skip-tz-utc` 添加这个参数忽略掉 TiDB 与导数据的机器之间时区设置不一致的情况,禁止自动转换。
添加 `--skip-tz-utc` 参数后,会忽略掉 TiDB 与导数据的机器之间时区设置不一致的情况,禁止自动转换。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

原句几乎读不通。


如果 `mydumper` 出现以下报错:

```
** (mydumper:27528): CRITICAL **: 13:25:09.081: Could not read data from testSchema.testTable: GC life time is shorter than transaction duration, transaction starts at 2019-08-05 21:10:01.451 +0800 CST, GC safe point is 2019-08-05 21:14:53.801 +0800 CST
```

就再执行两步命令:

1. 执行 `mydumper` 命令前,查询 TiDB 集群的 GC 值并使用 MySQL 客户端将其调整为合适的值。
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. 执行 `mydumper` 命令前,查询 TiDB 集群的 GC 值并使用 MySQL 客户端将其调整为合适的值
1. 执行 `mydumper` 命令前,查询 TiDB 集群的 [GC](/dev/reference/garbage-collection/overview.md) 值并使用 MySQL 客户端将其调整为合适的值

Copy link
Contributor

@TomShawn TomShawn Feb 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

读者可能不知道什么是 GC,这里最好能写全 Garbage Collection (GC),或至少向用户读者提供可参考的页面。


{{< copyable "sql" >}}

```sql
SELECT * FROM mysql.tidb WHERE VARIABLE_NAME = 'tikv_gc_life_time';
```

```
+-----------------------+------------------------------------------------------------------------------------------------+
| VARIABLE_NAME | VARIABLE_VALUE |
+-----------------------+------------------------------------------------------------------------------------------------+
| tikv_gc_life_time | 10m0s |
+-----------------------+------------------------------------------------------------------------------------------------+
1 rows in set (0.02 sec)
```

{{< copyable "sql" >}}

```sql
update mysql.tidb set VARIABLE_VALUE = '720h' where VARIABLE_NAME = 'tikv_gc_life_time';
```

2. 执行 `mydumper` 命令后,将 TiDB 集群的 GC 值恢复到第 1 步中的初始值。
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved

{{< copyable "sql" >}}

```sql
update mysql.tidb set VARIABLE_VALUE = '10m' where VARIABLE_NAME = 'tikv_gc_life_time';
```

## 向 TiDB 恢复数据

我们使用 `tidb lightning` 将之前导出的数据导入到 TiDB,完成恢复操作。具体的使用方法见 [tidb lightning 使用文档](/dev/reference/tools/tidb-lightning/tidb-backend.md)
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved
185 changes: 0 additions & 185 deletions dev/how-to/maintain/backup-and-restore/mydumper-loader.md

This file was deleted.

2 changes: 1 addition & 1 deletion dev/reference/tools/br/br.md
Expand Up @@ -6,7 +6,7 @@ category: how-to

# 使用 BR 进行备份与恢复

Backup & Restore(以下简称 BR)是 TiDB 分布式备份恢复的命令行工具,用于对 TiDB 集群进行数据备份和恢复。相比 [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md),BR 更适合大数据量的场景。本文档介绍了 BR 的使用限制、工作原理、命令行描述、备份恢复用例以及最佳实践。
Backup & Restore(以下简称 BR)是 TiDB 分布式备份恢复的命令行工具,用于对 TiDB 集群进行数据备份和恢复。相比 [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-lightning.md),BR 更适合大数据量的场景。本文档介绍了 BR 的使用限制、工作原理、命令行描述、备份恢复用例以及最佳实践。

## 使用限制

Expand Down
2 changes: 1 addition & 1 deletion dev/reference/tools/use-guide.md
Expand Up @@ -130,7 +130,7 @@ Loader 使用 SQL 的方式进行全量数据恢复,效率较低。TiDB Lightn

针对 TiDB 的 2.1,3.0 以及 3.1 版本,下面给出典型业务场景下的数据迁移方案。

### TiDB 2.1 和 3.0 全链路数据迁移方案
### TiDB 3.0 全链路数据迁移方案

#### MySQL 数据迁移到 TiDB

Expand Down
1 change: 0 additions & 1 deletion v2.1/TOC.md
Expand Up @@ -260,7 +260,6 @@
- [常见错误修复](/v2.1/reference/tidb-binlog/troubleshoot/error-handling.md)
- [FAQ](/v2.1/reference/tidb-binlog/faq.md)
+ 周边工具
- [工具使用指南](/v2.1/reference/tools/use-guide.md)
- [Mydumper](/v2.1/reference/tools/mydumper.md)
IANTHEREAL marked this conversation as resolved.
Show resolved Hide resolved
- [Loader](/v2.1/reference/tools/loader.md)
- [Syncer](/v2.1/reference/tools/syncer.md)
Expand Down