Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

op-guide, tools: Recommend mydumper from enterprise tools #644

Merged
merged 12 commits into from
Oct 11, 2018
14 changes: 1 addition & 13 deletions FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -691,24 +691,12 @@ TiDB is not suitable for tables of small size (such as below ten million level),

#### How to back up data in TiDB?

Currently, the major way of backing up data in TiDB is using `mydumper`. For details, see [mydumper repository](https://github.com/maxbube/mydumper). Although the official MySQL tool `mysqldump` is also supported in TiDB to back up and restore data, its performance is poorer than `mydumper`/`loader` and it needs much more time to back up and restore large volumes of data. Therefore, it is not recommended to use `mysqldump`.
Currently, the preferred method for backup is using the [PingCAP fork of mydumper](tools/mydumper.md). Although the official MySQL tool `mysqldump` is also supported in TiDB to back up and restore data, its performance is poorer than [`mydumper`](tools/mydumper.md)/[`loader`](tools/loader.md) and it needs much more time to back up and restore large volumes of data.

Keep the size of the data file exported from `mydumper` as small as possible. It is recommended to keep the size within 64M. You can set value of the `-F` parameter to 64.

You can edit the `t` parameter of `loader` based on the number of TiKV instances and load status. For example, in scenarios of three TiKV instances, you can set its value to `3 * (1 ~ n)`. When the TiKV load is very high and `backoffer.maxSleep 15000ms is exceeded` displays a lot in `loader` and TiDB logs, you can adjust the parameter to a smaller value. When the TiKV load is not very high, you can adjust the parameter to a larger value accordingly.

## Migrate the data and traffic
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we must keep "## Migrate the data and traffic" and "### Full data export and import". Otherwise, the following will have no corresponding title.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry! I misunderstood what this part meant. I will submit a new PR to restore.


### Full data export and import

#### Mydumper

See the [mydumper repository](https://github.com/maxbube/mydumper).

#### Loader

See [Loader Instructions](tools/loader.md).

#### How to migrate an application running on MySQL to TiDB?

Because TiDB supports most MySQL syntax, generally you can migrate your applications to TiDB without changing a single line of code in most cases. You can use [checker](https://github.com/pingcap/tidb-tools/tree/master/checker) to check whether the Schema in MySQL is compatible with TiDB.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@
- [Troubleshoot](trouble-shooting.md)
+ TiDB Enterprise Tools
- [Syncer](tools/syncer.md)
- [mydumper](tools/mydumper.md)
- [Loader](tools/loader.md)
- [TiDB-Binlog](tools/tidb-binlog-kafka.md)
- [PD Control](tools/pd-control.md)
Expand Down
6 changes: 3 additions & 3 deletions op-guide/backup-restore.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@ cd tidb-enterprise-tools-latest-linux-amd64

## Full backup and restoration using `mydumper`/`loader`

You can use `mydumper` to export data from MySQL and `loader` to import the data into TiDB.
You can use [`mydumper`](../tools/mydumper.md) to export data from MySQL and [`loader`](../tools/loader.md) to import the data into TiDB.

> **Note**: Although TiDB also supports the official `mysqldump` tool from MySQL for data migration, it is not recommended to use it. Its performance is much lower than `mydumper`/`loader` and it takes much time to migrate large amounts of data. `mydumper`/`loader` is more powerful. For more information, see https://github.com/maxbube/mydumper.
> **Important**: You must use the `mydumper` from the Enterprise Tools package, and not the `mydumper` provided by your operating system's package manager. The upstream version of `mydumper` does not yet handle TiDB correctly ([#155](https://github.com/maxbube/mydumper/pull/155)). Using `mysqldump` is also not recommended, as it is much slower for both backup and restoration.

### Best practices of full backup and restoration using `mydumper`/`loader`

Expand Down Expand Up @@ -119,4 +119,4 @@ mysql> select * from t2;
| 2 | b |
| 3 | c |
+----+------+
```
```
47 changes: 47 additions & 0 deletions tools/mydumper.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: mydumper Instructions
summary: Use mydumper to export data from TiDB.
category: tools
---

# mydumper Instructions

## What is mydumper?

`mydumper` is a fork of the [mydumper](https://github.com/maxbube/mydumper) project with additional functionality specific to TiDB. It is the recommended method to use for logical backups of TiDB.

[Download the Binary](http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.tar.gz).

## What enhancements does this contain over regular mydumper?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the following sentences, we had better use "Uses", "Includes", and "Allows" instead of "Use", "Include" and "Allow", because these sentences are descriptive (describe something) rather than imperative (tell users to do something).
Reference: "These comments should be descriptive ("Opens the file") rather than imperative ("Open the file"); the comment describes the function, it does not tell the function what to do." on https://google.github.io/styleguide/cppguide.html

+ Uses `tidb_snapshot` to provide backup consistency instead of `FLUSH TABLES WITH READ LOCK`

+ Includes the hidden `_tidb_rowid` column in `INSERT` statements when present

+ Allows `tidb_snapshot` to be [configurable](../op-guide/history-read.md#how-tidb-reads-data-from-history-versions) (i.e. backup data as it appeared at an earlier point in time)

### New parameter description

```
-z, --tidb-snapshot: Set the tidb_snapshot to be used for the backup.
Default: NOW()-INTERVAL 1 SECOND.
kennytm marked this conversation as resolved.
Show resolved Hide resolved
Accepts either a TSO or valid datetime. For example: -z "2016-10-08 16:45:26"
```

### Usage example

Command line parameter:

```
./bin/mydumper -h 127.0.0.1 -u root -P 4000
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a line after it.
Note: For a code block enclosed with "```", we should add a line both before and after it so that this block can be displayed normally on our website.


## FAQ

### Is the source code for these changes available?

Source code for PingCAP's mydumper is [available on GitHub](https://github.com/pingcap/mydumper).

### Do you plan to make these changes available to upstream mydumper?

Yes, we intend to make our changes available to upstream mydumper. See [PR #155](https://github.com/maxbube/mydumper/pull/155).