From ded10fb305a47e34acba5b48fdf182658a2c23aa Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 29 Nov 2019 11:55:19 +0800 Subject: [PATCH 01/20] tools: add br tool doc and refine mydumper/loader doc --- dev/TOC.md | 4 +- dev/how-to/maintain/backup-and-restore/br.md | 227 ++++++++++++++++++ .../mydumper-loader.md} | 82 ++++--- v3.1/TOC.md | 4 +- v3.1/how-to/maintain/backup-and-restore/br.md | 227 ++++++++++++++++++ .../mydumper-loader.md} | 82 ++++--- 6 files changed, 562 insertions(+), 64 deletions(-) create mode 100644 dev/how-to/maintain/backup-and-restore/br.md rename dev/how-to/maintain/{backup-and-restore.md => backup-and-restore/mydumper-loader.md} (51%) create mode 100644 v3.1/how-to/maintain/backup-and-restore/br.md rename v3.1/how-to/maintain/{backup-and-restore.md => backup-and-restore/mydumper-loader.md} (51%) diff --git a/dev/TOC.md b/dev/TOC.md index d4b9cd8a55140..33e94063867e6 100644 --- a/dev/TOC.md +++ b/dev/TOC.md @@ -82,7 +82,9 @@ - [Migrate from CSV](/dev/reference/tools/tidb-lightning/csv.md) + Maintain - [Common Ansible Operations](/dev/how-to/deploy/orchestrated/ansible-operations.md) - - [Backup and Restore](/dev/how-to/maintain/backup-and-restore.md) + + Backup and Restore + - [Use Mydumper/Loader](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md) + - [Use BR](/dev/how-to/maintain/backup-and-restore/br.md) - [Identify Slow Queries](/dev/how-to/maintain/identify-slow-queries.md) + Scale - [Scale using Ansible](/dev/how-to/scale/with-ansible.md) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md new file mode 100644 index 0000000000000..7958bcea7fd0b --- /dev/null +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -0,0 +1,227 @@ +--- +title: Use BR to Backup and Restore Cluster Data +summary: Learn how to backup and restore the data of the TiDB cluster using BR. +category: how-to +--- + +# Use BR to Backup and Restore Cluster Data + +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with Mydumper/Loader, BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases of this tool. + +## Working principle + +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding back and restoration operations. In an operation, each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. + +![br-arch](/media/br-arch.png) + +## Command-line description + +A `br` command consists of sub-commands, options and parameters. The sub-command is the characters without `-` or `--`. The option is the characters that start with `-` or `--`. The parameter is the characters that immediately follow and pass to the sub-command or the option. + +This is a complete `br` command: + +`br backup full --pd "${PDIP}:2379" -s "local:///tmp/backup"` + +In this command, + +* `backup` is the sub-command of `br`. +* `full` is the sub-command of `backup`. +* `-s` or `--storage` specifies the path in which the backup files are stored. +* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the storage path in the local disk. +* `--pd` is the PD service address. +* `"${PDIP}:2379"` is the parameter of `--pd`. + +### Sub-commands + +A `br` command consists of multiple layers of sub-command. Currently, BR has the following three sub-commands: + +* `br backup` is used to backup the data of the TiDB cluster. +* `br restore` is used to restore the data of the TiDB cluster. +* `br version` is used to check the version of the BR tool. + +Each of the above three sub-commands includes the following three available sub-commands: + +* `full` is used to backup or restore all data in the cluster. +* `db` is used to restore the specified database of the cluster. +* `table` is used to backup or restore a single table in the specified database of the cluster. + +### Common options + +* `--pd` is for connection, specifying the PD server address. For example, `"${PDIP}:2379"`. +* `-h`/`--help` is used to get help on all sub-commands. For example, `br backup --help`. +* `--ca` specifies the path to the trusted CA certificate in the PEM format. +* `--cert` specifies the path to the SSL certificate in the PEM format. +* `--key` specifies the path to the SSL certificate key in the PEM format. +* `--status-addr` specifies the listening address through which BR provides statistics to Prometheus. + +## Backup cluster data + +To backup the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table. + +### Backup all cluster data + +To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. + +Use case: Backup all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. + +{{< copyable "shell-regular" >}} + +```shell +br backup full \ + --pd "${PDIP}:2379" \ + --storage "local:///tmp/backup" \ + --ratelimit 120 \ + --concurrency 4 \ + --log-file backupfull.log +``` + +In the above `br` command, the `--ratelimit` and `--concurrency` options set upper limits on the speed at which a backup operation is performed (MiB/s) and the number of concurrent executions for each TiKV node, and the BR log is written to the `backupfull.log` file. + +A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: + +```shell +br backup full \ + --pd "${PDIP}:2379" \ + --storage "local:///tmp/backup" \ + --ratelimit 120 \ + --concurrency 4 \ + --log-file backupfull.log +Full Backup <---------/................................................> 17.12%. +``` + +### Backup single table + +To backup the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. + +Use case: Backup the data of the `test.usertable` table to the `/tmp/backup` path in each TiKV node and write the `backupmeta` file to this path. + +{{< copyable "shell-regular" >}} + +```shell +br backup table \ + --pd "${PDIP}:2379" \ + --db test \ + --table usertable \ + --storage "local:///tmp/backup" \ + --ratelimit 120 \ + --concurrency 4 \ + --log-file backuptable.log +``` + +The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. The other options have the same meanings as those in [Backup whole cluster](#backup-whole-cluster). + +A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then, the BR also checks the backup data to ensure data security. + +## Restore cluster data + +To restore the cluster data, use the `br restore` command. You can add the `full`, `db` or `table` sub-command to specify the scope of your restoration: the whole cluster, a database or a single table. + +### Restore all backup data + +To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. + +Usage: Restore all the backup data in the `/tmp/backup` path to the cluster. + +{{< copyable "shell-regular" >}} + +```shell +br restore full \ + --pd "${PDIP}:2379" \ + --storage "local:///tmp/backup" \ + --concurrency 128 \ + --log-file restorefull.log +``` + +`--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation, and the BR log is written to the `restorefull.log` file. + +A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then, the BR also checks the backup data to ensure data security. + +```shell +br restore full \ + --pd "${PDIP}:2379" \ + --storage "local:///tmp/backup" \ + --log-file restorefull.log +Full Restore <---------/...............................................> 17.12%. +``` + +### Restore a database + +To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. + +Usage: Restore a database in the backup data in the `/tmp/backup` path to the cluster. + +{{< copyable "shell-regular" >}} + +```shell +br restore db \ + --pd "${PDIP}:2379" \ + --db "test" \ + --storage "local:///tmp/backup" \ + --log-file restorefull.log +``` + +In the above command, `--db` specifies the name of the database to be restored. The other options have the same meaning as those in [Restore all backup data](#restore-all-backup-data). + +### Restore a table + +To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. + +Usage: Restore a database table in the backup data in the `/tmp/backup` path to the cluster. + +{{< copyable "shell-regular" >}} + +```shell +br restore table \ + --pd "${PDIP}:2379" \ + --db "test" \ + --table "usertable" \ + --storage "local:///tmp/backup" \ + --log-file restorefull.log +``` + +In the above command, `--table` specifies the name of the table to be restored. The other options have the same meaning as those in [Restore a database](#restore-a-database). + +## Best practices + +- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`. This makes it easier to collect and manage backup files. +- It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. +- It is recommended that you perform the backup operation in a low-peak application time to minimize the impact on the application. + +## Note + +- BR only supports TiDB v3.1 or later versions. +- If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. +- TiDB cannot perform the backup operation when executing DDL statements. +- The restoration can be performed only on new clusters. +- If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `"10m0s"` by default, increase the value of this configuration. + + For example, set `tikv_gc_life_time` to `720h`: + + {{< copyable "sql" >}} + + ``` + mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ + "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; + ``` + +- To speed up the restoration, use pd-ctl to remove the schedulers related to scheduling before the restoration and add back these removed schedulers after the restoration. + + Remove schedulers: + + {{< copyable "shell-regular" >}} + + ```shell + ./pd-ctl -u ${PDIP}:2379 scheduler remove balance-hot-region-scheduler + ./pd-ctl -u ${PDIP}:2379 scheduler remove balance-leader-scheduler + ./pd-ctl -u ${PDIP}:2379 scheduler remove balance-region-scheduler + ``` + + Add schedulers: + + {{< copyable "shell-regular" >}} + + ```shell + ./pd-ctl -u ${PDIP}:2379 scheduler add balance-hot-region-scheduler + ./pd-ctl -u ${PDIP}:2379 scheduler add balance-leader-scheduler + ./pd-ctl -u ${PDIP}:2379 scheduler add balance-region-scheduler + ``` diff --git a/dev/how-to/maintain/backup-and-restore.md b/dev/how-to/maintain/backup-and-restore/mydumper-loader.md similarity index 51% rename from dev/how-to/maintain/backup-and-restore.md rename to dev/how-to/maintain/backup-and-restore/mydumper-loader.md index d9aab57c1187d..9a549219a6c01 100644 --- a/dev/how-to/maintain/backup-and-restore.md +++ b/dev/how-to/maintain/backup-and-restore/mydumper-loader.md @@ -1,14 +1,15 @@ --- -title: Backup and Restore -summary: Learn how to back up and restore the data of TiDB. +title: Use `mydumper`/`loader` to Backup and Restore +summary: Learn how to backup and restore the data of TiDB using `mydumper`/`loader`. category: how-to +aliases: ['/docs/dev/how-to/maintain/backup-and-restore/'] --- -# Backup and Restore +# Use `mydumper`/`loader` to Backup and Restore -This document describes how to back up and restore the data of TiDB. Currently, this document only covers full backup and restoration. +This document describes how to back up and restore the data of TiDB using `mydumper`/`loader`. Currently, this document only covers full backup and restoration. -Here we assume that the TiDB service information is as follows: +Suppose that the TiDB service information is as follows: |Name|Address|Port|User|Password| |:----:|:-------:|:----:|:----:|:------:| @@ -21,33 +22,48 @@ Use the following tools for data backup and restoration: ## Download TiDB toolset (Linux) -```bash -# Download the tool package. -wget http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.tar.gz -wget http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.sha256 +1. Download the tool package: -# Check the file integrity. If the result is OK, the file is correct. -sha256sum -c tidb-enterprise-tools-latest-linux-amd64.sha256 + {{< copyable "shell-regular" >}} -# Extract the package. -tar -xzf tidb-enterprise-tools-latest-linux-amd64.tar.gz -cd tidb-enterprise-tools-latest-linux-amd64 -``` + ```bash + wget http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.tar.gz + wget http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.sha256 + ``` + +2. Check the file integrity. If the result is fine, the file is correct. + + {{< copyable "shell-regular" >} + + ```bash + sha256sum -c tidb-enterprise-tools-latest-linux-amd64.sha256 + ``` + +3. Extract the package: + + {{< copyable "shell-regular" >} + + ```bash + tar -xzf tidb-enterprise-tools-latest-linux-amd64.tar.gz + cd tidb-enterprise-tools-latest-linux-amd64 + ``` ## Full backup and restoration using `mydumper`/`loader` -You can use [`mydumper`](/dev/reference/tools/mydumper.md) to export data from TiDB and [`loader`](/dev/reference/tools/loader.md) to import the data into TiDB. +Use [`mydumper`](/dev/reference/tools/mydumper.md) to export data from TiDB and [`loader`](/dev/reference/tools/loader.md) to import the data into TiDB. -> **Important**: You must use the `mydumper` from the Enterprise Tools package, and not the `mydumper` provided by your operating system's package manager. The upstream version of `mydumper` does not yet handle TiDB correctly ([#155](https://github.com/maxbube/mydumper/pull/155)). Using `mysqldump` is also not recommended, as it is much slower for both backup and restoration. +> **Note:** +> +> Use `mydumper` from the Enterprise Tools package, and not the `mydumper` provided by your operating system's package manager. The upstream version of `mydumper` does not yet handle TiDB correctly ([#155](https://github.com/maxbube/mydumper/pull/155)). It is also not recommended to use `mysqldump` which is much slower for both backup and restoration. -### Best practices of full backup and restoration using `mydumper`/`loader` +### Best practices for full backup and restoration using `mydumper`/`loader` To quickly backup and restore data (especially large amounts of data), refer to the following recommendations: -- Keep the exported data file as small as possible and it is recommended keep it within 64M. You can use the `-F` parameter to set the value. -- You can adjust the `-t` parameter of `loader` based on the number and the load of TiKV instances. For example, if there are three TiKV instances, `-t` can be set to around 3 * (1 ~ n). If the load of TiKV is too high and the log `backoffer.maxSleep 15000ms is exceeded` is displayed many times, decrease the value of `-t`; otherwise, increase it. +- Keep the exported data file as small as possible and it is recommended to keep it smaller than 64M. Use the `-F` parameter to set the value. +- Adjust the `-t` parameter of `loader` based on the number and the load of TiKV instances. For example, if there are three TiKV instances, `-t` can be set to around `3 * (1 ~ n)`. If the load of TiKV is too high and the `backoffer.maxSleep 15000ms is exceeded` log is displayed many times, decrease the value of `-t`; otherwise, increase the value. -#### An example of restoring data and related configuration +#### An example of restoring data and related configurations - The total size of the exported files is 214G. A single table has 8 columns and 2 billion rows. - The cluster topology: @@ -60,27 +76,29 @@ To quickly backup and restore data (especially large amounts of data), refer to - Memory: 128G - Disk: sda [raid 10, 300G] sdb[RAID 5, 2T] - Operating System: CentOS 7.3 -- The `-F` parameter of `mydumper` is set to 16 and the `-t` parameter of `loader` is set to 64. +- The `-F` parameter of `mydumper` is set to `16` and the `-t` parameter of `loader` is set to `64`. -**Results**: It takes 11 hours to import all the data, which is 19.4G/hour. +Results: It takes 11 hours to import all the data, which is 19.4G/hour. ### Backup data from TiDB Use `mydumper` to backup data from TiDB. +{{< copyable "shell-regular" >}} + ```bash ./bin/mydumper -h 127.0.0.1 -P 4000 -u root -t 16 -F 64 -B test -T t1,t2 --skip-tz-utc -o ./var/test ``` In this command, -- `-B test`: means the data is exported from the `test` database. -- `-T t1,t2`: means only the `t1` and `t2` tables are exported. -- `-t 16`: means 16 threads are used to export the data. -- `-F 64`: means a table is partitioned into chunks and one chunk is 64MB. -- `--skip-tz-utc`: the purpose of adding this parameter is to ignore the inconsistency of time zone setting between MySQL and the data exporting machine and to disable automatic conversion. +- `-B test` means that the data is exported from the `test` database. +- `-T t1,t2` means that only the `t1` and `t2` tables are exported. +- `-t 16` means that 16 threads are used to export the data. +- `-F 64` means that a table is partitioned into chunks and one chunk is 64MB. +- `--skip-tz-utc` means to ignore the inconsistency of time zone setting between MySQL and the data exporting machine and to disable automatic conversion. -If `mydumper` emits error like: +If `mydumper` returns the following error: ``` ** (mydumper:27528): CRITICAL **: 13:25:09.081: Could not read data from testSchema.testTable: GC life time is shorter than transaction duration, transaction starts at 2019-08-05 21:10:01.451 +0800 CST, GC safe point is 2019-08-05 21:14:53.801 +0800 CST @@ -88,7 +106,7 @@ If `mydumper` emits error like: Then execute two more commands: -- Step 1: before executing the `mydumper` command, query the GC values of the TiDB cluster and adjust it to a suitable value using the MySQL client. +1. Before executing the `mydumper` command, query the GC values of the TiDB cluster and adjust it to a suitable value using the MySQL client: ```sql mysql> SELECT * FROM mysql.tidb WHERE VARIABLE_NAME = 'tikv_gc_life_time'; @@ -102,7 +120,7 @@ Then execute two more commands: mysql> update mysql.tidb set VARIABLE_VALUE = '720h' where VARIABLE_NAME = 'tikv_gc_life_time'; ``` -- Step 2: after you finish running the `mydumper` command, restore the GC value of the TiDB cluster to its original value in step 1. +2. After finishing running the `mydumper` command, restore the adjusted GC value (`720h`) of the TiDB cluster to its original value (`10m0s`) in step 1. {{< copyable "sql" >}} @@ -114,6 +132,8 @@ Then execute two more commands: To restore data into TiDB, use `loader` to import the previously exported data. See [Loader instructions](/dev/reference/tools/loader.md) for more information. +{{< copyable "shell-regular" >}} + ```bash ./bin/loader -h 127.0.0.1 -u root -P 4000 -t 32 -d ./var/test ``` diff --git a/v3.1/TOC.md b/v3.1/TOC.md index c42b22c4a8734..028bf1f78f270 100644 --- a/v3.1/TOC.md +++ b/v3.1/TOC.md @@ -81,7 +81,9 @@ - [Migrate from CSV](/v3.1/reference/tools/tidb-lightning/csv.md) + Maintain - [Common Ansible Operations](/v3.1/how-to/deploy/orchestrated/ansible-operations.md) - - [Backup and Restore](/v3.1/how-to/maintain/backup-and-restore.md) + + Backup and Restore + - [Use Mydumper/Loader](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md) + - [Use BR](/v3.1/how-to/maintain/backup-and-restore/br.md) - [Identify Slow Queries](/v3.1/how-to/maintain/identify-slow-queries.md) + Scale - [Scale using Ansible](/v3.1/how-to/scale/with-ansible.md) diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md new file mode 100644 index 0000000000000..ca72350fb5a53 --- /dev/null +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -0,0 +1,227 @@ +--- +title: Use BR to Backup and Restore Cluster Data +summary: Learn how to backup and restore the data of the TiDB cluster using BR. +category: how-to +--- + +# Use BR to Backup and Restore Cluster Data + +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with Mydumper/Loader, BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases of this tool. + +## Working principle + +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding back and restoration operations. In an operation, each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. + +![br-arch](/media/br-arch.png) + +## Command-line description + +A `br` command consists of sub-commands, options and parameters. The sub-command is the characters without `-` or `--`. The option is the characters that start with `-` or `--`. The parameter is the characters that immediately follow and pass to the sub-command or the option. + +This is a complete `br` command: + +`br backup full --pd "${PDIP}:2379" -s "local:///tmp/backup"` + +In this command, + +* `backup` is the sub-command of `br`. +* `full` is the sub-command of `backup`. +* `-s` or `--storage` specifies the path in which the backup files are stored. +* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the storage path in the local disk. +* `--pd` is the PD service address. +* `"${PDIP}:2379"` is the parameter of `--pd`. + +### Sub-commands + +A `br` command consists of multiple layers of sub-command. Currently, BR has the following three sub-commands: + +* `br backup` is used to backup the data of the TiDB cluster. +* `br restore` is used to restore the data of the TiDB cluster. +* `br version` is used to check the version of the BR tool. + +Each of the above three sub-commands includes the following three available sub-commands: + +* `full` is used to backup or restore all data in the cluster. +* `db` is used to restore the specified database of the cluster. +* `table` is used to backup or restore a single table in the specified database of the cluster. + +### Common options + +* `--pd` is for connection, specifying the PD server address. For example, `"${PDIP}:2379"`. +* `-h`/`--help` is used to get help on all sub-commands. For example, `br backup --help`. +* `--ca` specifies the path to the trusted CA certificate in the PEM format. +* `--cert` specifies the path to the SSL certificate in the PEM format. +* `--key` specifies the path to the SSL certificate key in the PEM format. +* `--status-addr` specifies the listening address through which BR provides statistics to Prometheus. + +## Backup cluster data + +To backup the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table. + +### Backup all cluster data + +To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. + +Use case: Backup all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. + +{{< copyable "shell-regular" >}} + +```shell +br backup full \ + --pd "${PDIP}:2379" \ + --storage "local:///tmp/backup" \ + --ratelimit 120 \ + --concurrency 4 \ + --log-file backupfull.log +``` + +In the above `br` command, the `--ratelimit` and `--concurrency` options set upper limits on the speed at which a backup operation is performed (MiB/s) and the number of concurrent executions for each TiKV node, and the BR log is written to the `backupfull.log` file. + +A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: + +```shell +br backup full \ + --pd "${PDIP}:2379" \ + --storage "local:///tmp/backup" \ + --ratelimit 120 \ + --concurrency 4 \ + --log-file backupfull.log +Full Backup <---------/................................................> 17.12%. +``` + +### Backup single table + +To backup the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. + +Use case: Backup the data of the `test.usertable` table to the `/tmp/backup` path in each TiKV node and write the `backupmeta` file to this path. + +{{< copyable "shell-regular" >}} + +```shell +br backup table \ + --pd "${PDIP}:2379" \ + --db test \ + --table usertable \ + --storage "local:///tmp/backup" \ + --ratelimit 120 \ + --concurrency 4 \ + --log-file backuptable.log +``` + +The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. The other options have the same meanings as those in [Backup whole cluster](#backup-whole-cluster). + +A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then, the BR also checks the backup data to ensure data security. + +## Restore cluster data + +To restore the cluster data, use the `br restore` command. You can add the `full`, `db` or `table` sub-command to specify the scope of your restoration: the whole cluster, a database or a single table. + +### Restore all backup data + +To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. + +Usage: Restore all the backup data in the `/tmp/backup` path to the cluster. + +{{< copyable "shell-regular" >}} + +```shell +br restore full \ + --pd "${PDIP}:2379" \ + --storage "local:///tmp/backup" \ + --concurrency 128 \ + --log-file restorefull.log +``` + +`--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation, and the BR log is written to the `restorefull.log` file. + +A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then, the BR also checks the backup data to ensure data security. + +```shell +br restore full \ + --pd "${PDIP}:2379" \ + --storage "local:///tmp/backup" \ + --log-file restorefull.log +Full Restore <---------/...............................................> 17.12%. +``` + +### Restore a database + +To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. + +Usage: Restore a database in the backup data in the `/tmp/backup` path to the cluster. + +{{< copyable "shell-regular" >}} + +```shell +br restore db \ + --pd "${PDIP}:2379" \ + --db "test" \ + --storage "local:///tmp/backup" \ + --log-file restorefull.log +``` + +In the above command, `--db` specifies the name of the database to be restored. The other options have the same meaning as those in [Restore all backup data](#restore-all-backup-data). + +### Restore a table + +To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. + +Usage: Restore a database table in the backup data in the `/tmp/backup` path to the cluster. + +{{< copyable "shell-regular" >}} + +```shell +br restore table \ + --pd "${PDIP}:2379" \ + --db "test" \ + --table "usertable" \ + --storage "local:///tmp/backup" \ + --log-file restorefull.log +``` + +In the above command, `--table` specifies the name of the table to be restored. The other options have the same meaning as those in [Restore a database](#restore-a-database). + +## Best practices + +- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`. This makes it easier to collect and manage backup files. +- It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. +- It is recommended that you perform the backup operation in a low-peak application time to minimize the impact on the application. + +## Note + +- BR only supports TiDB v3.1 or later versions. +- If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. +- TiDB cannot perform the backup operation when executing DDL statements. +- The restoration can be performed only on new clusters. +- If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `"10m0s"` by default, increase the value of this configuration. + + For example, set `tikv_gc_life_time` to `720h`: + + {{< copyable "sql" >}} + + ``` + mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ + "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; + ``` + +- To speed up the restoration, use pd-ctl to remove the schedulers related to scheduling before the restoration and add back these removed schedulers after the restoration. + + Remove schedulers: + + {{< copyable "shell-regular" >}} + + ```shell + ./pd-ctl -u ${PDIP}:2379 scheduler remove balance-hot-region-scheduler + ./pd-ctl -u ${PDIP}:2379 scheduler remove balance-leader-scheduler + ./pd-ctl -u ${PDIP}:2379 scheduler remove balance-region-scheduler + ``` + + Add schedulers: + + {{< copyable "shell-regular" >}} + + ```shell + ./pd-ctl -u ${PDIP}:2379 scheduler add balance-hot-region-scheduler + ./pd-ctl -u ${PDIP}:2379 scheduler add balance-leader-scheduler + ./pd-ctl -u ${PDIP}:2379 scheduler add balance-region-scheduler + ``` diff --git a/v3.1/how-to/maintain/backup-and-restore.md b/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md similarity index 51% rename from v3.1/how-to/maintain/backup-and-restore.md rename to v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md index 788c21e43b4a1..a018dac40591c 100644 --- a/v3.1/how-to/maintain/backup-and-restore.md +++ b/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md @@ -1,14 +1,15 @@ --- -title: Backup and Restore -summary: Learn how to back up and restore the data of TiDB. +title: Use `mydumper`/`loader` to Backup and Restore +summary: Learn how to backup and restore the data of TiDB using `mydumper`/`loader`. category: how-to +aliases: ['/docs/v3.1/how-to/maintain/backup-and-restore/'] --- -# Backup and Restore +# Use `mydumper`/`loader` to Backup and Restore -This document describes how to back up and restore the data of TiDB. Currently, this document only covers full backup and restoration. +This document describes how to back up and restore the data of TiDB using `mydumper`/`loader`. Currently, this document only covers full backup and restoration. -Here we assume that the TiDB service information is as follows: +Suppose that the TiDB service information is as follows: |Name|Address|Port|User|Password| |:----:|:-------:|:----:|:----:|:------:| @@ -21,33 +22,48 @@ Use the following tools for data backup and restoration: ## Download TiDB toolset (Linux) -```bash -# Download the tool package. -wget http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.tar.gz -wget http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.sha256 +1. Download the tool package: -# Check the file integrity. If the result is OK, the file is correct. -sha256sum -c tidb-enterprise-tools-latest-linux-amd64.sha256 + {{< copyable "shell-regular" >}} -# Extract the package. -tar -xzf tidb-enterprise-tools-latest-linux-amd64.tar.gz -cd tidb-enterprise-tools-latest-linux-amd64 -``` + ```bash + wget http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.tar.gz + wget http://download.pingcap.org/tidb-enterprise-tools-latest-linux-amd64.sha256 + ``` + +2. Check the file integrity. If the result is fine, the file is correct. + + {{< copyable "shell-regular" >} + + ```bash + sha256sum -c tidb-enterprise-tools-latest-linux-amd64.sha256 + ``` + +3. Extract the package: + + {{< copyable "shell-regular" >} + + ```bash + tar -xzf tidb-enterprise-tools-latest-linux-amd64.tar.gz + cd tidb-enterprise-tools-latest-linux-amd64 + ``` ## Full backup and restoration using `mydumper`/`loader` -You can use [`mydumper`](/v3.1/reference/tools/mydumper.md) to export data from TiDB and [`loader`](/v3.1/reference/tools/loader.md) to import the data into TiDB. +Use [`mydumper`](/v3.1/reference/tools/mydumper.md) to export data from TiDB and [`loader`](/v3.1/reference/tools/loader.md) to import the data into TiDB. -> **Important**: You must use the `mydumper` from the Enterprise Tools package, and not the `mydumper` provided by your operating system's package manager. The upstream version of `mydumper` does not yet handle TiDB correctly ([#155](https://github.com/maxbube/mydumper/pull/155)). Using `mysqldump` is also not recommended, as it is much slower for both backup and restoration. +> **Note:** +> +> Use `mydumper` from the Enterprise Tools package, and not the `mydumper` provided by your operating system's package manager. The upstream version of `mydumper` does not yet handle TiDB correctly ([#155](https://github.com/maxbube/mydumper/pull/155)). It is also not recommended to use `mysqldump` which is much slower for both backup and restoration. -### Best practices of full backup and restoration using `mydumper`/`loader` +### Best practices for full backup and restoration using `mydumper`/`loader` To quickly backup and restore data (especially large amounts of data), refer to the following recommendations: -- Keep the exported data file as small as possible and it is recommended keep it within 64M. You can use the `-F` parameter to set the value. -- You can adjust the `-t` parameter of `loader` based on the number and the load of TiKV instances. For example, if there are three TiKV instances, `-t` can be set to around 3 * (1 ~ n). If the load of TiKV is too high and the log `backoffer.maxSleep 15000ms is exceeded` is displayed many times, decrease the value of `-t`; otherwise, increase it. +- Keep the exported data file as small as possible and it is recommended to keep it smaller than 64M. Use the `-F` parameter to set the value. +- Adjust the `-t` parameter of `loader` based on the number and the load of TiKV instances. For example, if there are three TiKV instances, `-t` can be set to around `3 * (1 ~ n)`. If the load of TiKV is too high and the `backoffer.maxSleep 15000ms is exceeded` log is displayed many times, decrease the value of `-t`; otherwise, increase the value. -#### An example of restoring data and related configuration +#### An example of restoring data and related configurations - The total size of the exported files is 214G. A single table has 8 columns and 2 billion rows. - The cluster topology: @@ -60,27 +76,29 @@ To quickly backup and restore data (especially large amounts of data), refer to - Memory: 128G - Disk: sda [raid 10, 300G] sdb[RAID 5, 2T] - Operating System: CentOS 7.3 -- The `-F` parameter of `mydumper` is set to 16 and the `-t` parameter of `loader` is set to 64. +- The `-F` parameter of `mydumper` is set to `16` and the `-t` parameter of `loader` is set to `64`. -**Results**: It takes 11 hours to import all the data, which is 19.4G/hour. +Results: It takes 11 hours to import all the data, which is 19.4G/hour. ### Backup data from TiDB Use `mydumper` to backup data from TiDB. +{{< copyable "shell-regular" >}} + ```bash ./bin/mydumper -h 127.0.0.1 -P 4000 -u root -t 16 -F 64 -B test -T t1,t2 --skip-tz-utc -o ./var/test ``` In this command, -- `-B test`: means the data is exported from the `test` database. -- `-T t1,t2`: means only the `t1` and `t2` tables are exported. -- `-t 16`: means 16 threads are used to export the data. -- `-F 64`: means a table is partitioned into chunks and one chunk is 64MB. -- `--skip-tz-utc`: the purpose of adding this parameter is to ignore the inconsistency of time zone setting between MySQL and the data exporting machine and to disable automatic conversion. +- `-B test` means that the data is exported from the `test` database. +- `-T t1,t2` means that only the `t1` and `t2` tables are exported. +- `-t 16` means that 16 threads are used to export the data. +- `-F 64` means that a table is partitioned into chunks and one chunk is 64MB. +- `--skip-tz-utc` means to ignore the inconsistency of time zone setting between MySQL and the data exporting machine and to disable automatic conversion. -If `mydumper` emits error like: +If `mydumper` returns the following error: ``` ** (mydumper:27528): CRITICAL **: 13:25:09.081: Could not read data from testSchema.testTable: GC life time is shorter than transaction duration, transaction starts at 2019-08-05 21:10:01.451 +0800 CST, GC safe point is 2019-08-05 21:14:53.801 +0800 CST @@ -88,7 +106,7 @@ If `mydumper` emits error like: Then execute two more commands: -- Step 1: before executing the `mydumper` command, query the GC values of the TiDB cluster and adjust it to a suitable value using the MySQL client. +1. Before executing the `mydumper` command, query the GC values of the TiDB cluster and adjust it to a suitable value using the MySQL client: ```sql mysql> SELECT * FROM mysql.tidb WHERE VARIABLE_NAME = 'tikv_gc_life_time'; @@ -102,7 +120,7 @@ Then execute two more commands: mysql> update mysql.tidb set VARIABLE_VALUE = '720h' where VARIABLE_NAME = 'tikv_gc_life_time'; ``` -- Step 2: after you finish running the `mydumper` command, restore the GC value of the TiDB cluster to its original value in step 1. +2. After finishing running the `mydumper` command, restore the adjusted GC value (`720h`) of the TiDB cluster to its original value (`10m0s`) in step 1. {{< copyable "sql" >}} @@ -114,6 +132,8 @@ Then execute two more commands: To restore data into TiDB, use `loader` to import the previously exported data. See [Loader instructions](/v3.1/reference/tools/loader.md) for more information. +{{< copyable "shell-regular" >}} + ```bash ./bin/loader -h 127.0.0.1 -u root -P 4000 -t 32 -d ./var/test ``` From c4b9c049532ce72ea4a4c7032454ea165d32ac57 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 29 Nov 2019 15:03:19 +0800 Subject: [PATCH 02/20] tools: add diagram --- media/br-arch.png | Bin 0 -> 31354 bytes 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 media/br-arch.png diff --git a/media/br-arch.png b/media/br-arch.png new file mode 100644 index 0000000000000000000000000000000000000000..549b112f62b0652628802712647fc17ee955ab85 GIT binary patch literal 31354 zcmc$_XH=6<^adC#Ac~+Oh9*(zM5>5%lp-PsBE2^eklsNc0a3c3l+e5MCP;5VX%dj$ zdkc{oNN6F1?91=J`~R}%?AZ@{&VGQvn|WvM%-r(aXYPc*R#(1$mH8?N1iG&BLh%g< zbm=+>bfM@9CGce8-k2cpLGALwz#Rm-Mt}Z80ZK_@1RhejzfqP4mG`r*1OHsIdG_iV z2viw!?bPBj2=swYMe&)o55+c#F5Y3n1aiEy(|Tgn6Ir!Uuc-ONQc>y$dpet`m|pYw!O>|zZ~kiDG1=f3jSxcK@|ETH znT$%a#mmp#b~AnHKfauNdd1vCWgI@YBZ1ub>6}^aSKQdxR7^}O9{M$TP(vD0tO~C zFpRT12W*xjdCQdxozkxlML{>xJ?^T9H>ilMrNCMhx`! zVsZt#I>ZD?P&g;Qv9C39%d>StRS=AyFeskwN}5P^7M$bBKpxv8$Kl(5JSXzl)B8V@ zdS}8X5+f$8=J6UbkH?i#IxUfwWJH9u6$D?ED+|e*`vmB$P{$^Gk?JHylNYwO(}bc^`S3tHt-HiJl1F0|6G5_efO=mFc;1gt+zD81nI;h{9I4 zr2#C$b5v)meT-YT(IE_S7~E@Phj5Fml9@0^GoqH{M@w)He%Au(|3zNk4L2YzNkFqtZ_ z-CO5xXZW+=1e~~yGAALWiuDHgLeA_h@ME`YJ;%Nk0dh*aUwhetgP!?p7ZsdX7xSp@ z8%fHy0f|Ei9goM^npu=jyhoKhiJH8K+qQc~l$e|< z9|}_Zd}iA=`eKa}4;thkjCafxo6>~iKctPA>ACM&3FegEKB-fsEwWXX;7Q~bAWT=3 zUB7P1VjC{e_rr*J%9OyZTl*!3VevG3y3nNFpT2SK`# zEB|#0tD&jV(-iC33>QIJ{w+fUBlzmH)Q7tSWD`59`rbIEH)z-1xcn|=FtGA}=>V?g-orO7F%Jrk<{S`S`5AlEL+KG`v zX{JoUpZ6G&lbkyj!Ae5LBK_V`*!S%9QK2fq>>1hW1=k)tZ)^J|MPQp|sA=v|c z1ap>C4C9sm)M9)i{2DbO2bH&Q{H5luz>{aK`tC68LgBZ6-e-~S(5p?Ei$L5&kb0gy zr|KhJY!@R)jB0CiRuE7iDO^OgT)ca?_r6n2ADH98>o9YRL2>ZIoc4c%f`ToZCJGq) z-`h#TX`XY~DM3JyWy}w+Tu&-)d@GyDy0Tg|0o{^Frb|TR@`!!|BuruFYtcNMv@4#c zG-x@#$+tg}fKi;0h@PcaNal_+VrKAOvmym>iDX>Mz(jB#BZGMuXtY5eYExO|_VFysDWIp6SlG z7Xrs=oK$p|B{V!>!*_(eQI3r8j`|Vb9>mI_tsEAyjjb~)YzY(5n^k8b-U^WD5dG)x z`iwh9n*p7wM=u1eKKI6imT!P+`sn~8un>5khcSDm$~Pmms6zK(m#8$I{%l5E{Xk3e zfX64_hx|hzaMheW(Z2Y>L!TWR7?%wGr<~tX4g1OakI7YEZJXst{n*(Tp^L@K9f{Z; zh~DF595-1w;&lAxelIu7Y=PkJkqkJcp z%}!~IU&}rY2zG-cLq36OW88S%VJ7lTzxNdUEVCfb(yuZ`@NBvw?_jx-Z%yuv1N$0v zB^W&2ZtxH%#GIO{mPi_`sj^Ayky`MJAir2^G$TM56$2+ zjx+Ls#bDTh9#QKK{&`*_-|xVr-(d2doH~YzZ!8hHX|j$wYA*0%(Yu|n4vsvS(CNsH zJv)IM2VL6?n5%>PA#oWk$A8#)6SALOx~zAHh=3k~<1L<-l0VbIc5}YoZz`Q6PmmYs zoEEw*r}QzA5W)-OFa+sms+T^!@YA?keTHgU#kMV)18+p+0lbt~xX6q0c<1I_`45DJ z(^QxQJBnN4QkW4h(K@s+3v3ESNnArWNMtS$t>bVtzwllUGo8CjXP(E?TS9XD8G>W8_=RgEV&E82vnvqsE|h>Ob`aW`*>>g|v6rOm4!q+Rjy;>HtOyhD9$3 zqvSNj!jDXW(0(KrtZ7NMY;A@MmX#f3#{3L8_l8j%|0!gk*9Q}W@Iop6V{#N1h_jDoE~oKjrX`!1M5SD%nB!yqumMO4>j>-rf8L_+tN@`#>$q+mpq$ zAL3d0;6g9}z&i1rpJRSq_18YmrO`a{lQxULUoJ2ogVT-ow2XLa;X=5Empfjp-F-05 zf==p1xoM^!%K)7bk2@_>$F?zx9JQpzoW4hVtQ6Bq%3(AN+@U2X?dD9bMDAWaK0Wfv z>J9FYt(vOK@&}3x+%>0$CL);%;f;=~Z(AF`NPhFjwA~xmN{Az{GIqc^#B!=KHR?$V z2chtKuQ`e2fdta=T`M^&h!sIR$h1MhCZ-ZTw(zJ{X6KM1uq8a=g8-zX!7m_-^_Cpw zo_V&zLA}w~>*eH3(o`fL5eVG(<>RSC!ULjE$BU9DWvsoPI=yr{W)D9GZdC;>u-Ae9 z%rKS?Do?F8EMMp|9~9a;Dy5_}@Q?N-^fu02>Z9d9N3qD7)6azS@Xj2Jj*ui>qd+&n zcYn95FCPXQGsSq8(lyKpnQ8Y}f2rrpD)(IKRWO8heL(Ii$i+XOs@=xj3B5)%u(LDJ z<0>&xR@!nulTqe})LE|@Jqsm8KPB4QJ>~B?Fjg6&POQ;CEYMC5*|C{Qd~*uh^!uk=)VO2&n0cl=^$n*-atHQ` zXfT3uG`261$O|a5ZlsG`opIFSy?mZJ#m!D0IhxJyzO0c6Ztd!ug{=_lV7-x@PPZZD zVRWZbhz04@m_g~RWdnFjHn*Htw%0Xrsu$=GfE0!j$uSWQY=nE6`X%E>KCyF-BAN6% zFUlsAjmIF$H4L5V`QQ2`m|v_N5v4AGQgN3*!OV97O}=-!$VXs8u?Ft#vid+WmZzt#AuqxG> zn0$-6=R3EVAJSGA&PJviy1Z+0{Crh3aWF6&NHW?=!*PG1!U^n;J6~uHW$(Qy7{Si` zL!)>!S4yFggG=a}VEN*s;|gq*yh+QPYe8%LMzdjjfIuXSYJNYz8oc?Kq)vfX_|FNP z!|Ytf1RfOs5@4d`=@^0;Y3y3=Jtdc~UN@N)GZlv#EbG5d_7}D-;i`VFNZ&qiTAkr+ zH#PL_;?XI(qL_o$%WWE8xz&`0^Q_>t1rIKo^x zwCQc=yeFMhGF_b7ns zFRu+crxqIsuFZ^LD?>MM-W;O+)e zG30FNnjBs2?ZS{Nu0b4nAtlcgjN|{zgn8=J-NXYq-U*Ppkd5zFeRyI@+OiRphMdYQ zdxzns)w7=o#C4noGZ4@0J0+j%9O7?`X>vvQZp!xB!*JZhc9-rmwVVX5K^iBGEZv$| z=~AW{j#z@UWV>pNw40{fMT_Ac|GAcsCY80#6tU>IzV0&Cn;nXqQH{F~WP5ooNt>m8 zJ(_Od0R1qDf3iHwjF}WS>t0Dvx-5(0*BmeMie^n9{PH^}K_}Edb=yAofHHq5(HbXNWukle`f{B3k}!?KoL!(TBH!=<9T7 z5!{xoSNKhfnKmTl!;fi9i>fc(t);6>_t&)EJ?hJBH&sr*kYaa#$OnFszR>WE_L;U* z39nZgO$aC6lY6=lL?^ah!G*uihx*p1H?r;;N7y*)40oC=yLR(x)Ky9V2@#e#z}!v6 zJ~G_+cI`NV9I+fb?F!j|4ft#)j}*|66p5V3)86Q7AhlVG1e@aJX20*l+)vZoyB+zH zdqsuGh3=l7EW%je#h>9vdZUn`;>mu~3A4FwymjWw`;syJB<-;Tn~*XqLnH>qGtjnL zFB6oV;|8g_q-gzw@1JHvS;%Egm6#vq`qnbBgj`p&zoNAi_ps+o*voxYd;9gzX2hC+ z^iOW??UT~xWeSYalXGw^XT)b`8X z2@r43yeQ!V$27T0AMZzj%s=Ev?`^O-dDE;2)MYwG*;jxW-I#19WlgW#>e44aJI#3d z=*6PpQxQ3dSVFqW8bwkFSG>)d!R#TSarpwU!Ni|6ByJt&YhPgZmUCXF@&c2wwQJMo zzyo`o)l%+1;0J`0TzDQB4&WB+9G;nn33u6BcjmA=17NTsC9(LzTIwLkOGsTLQ%XK+ zh8F}%;c}vqG4z_37RSlgX5M5AoZ=J>ybI*cfwhrtFQXoOBU(rfuN(g}<3))HyUEYS z4t_V{3VL^gxsCQe2NxDyz8PY2iSfBDNa|}Jz#!pY+JdZN|NHBm@HxO7z43pcK;WOJ z{MqNvUaQ&yzd@t_3lRU`eC6`XJ<&K8By!%v;1QJE1CKrHM(!tGqH4Q*&Ru0;4=P%x z;iv7;3N_HP^8xk)(D;EP2zd+FOaJsgF#?=FtmmCsVK++t_rXK?;NJdB?2+9rZurfG z|7mrTvANR0aTnvH35q@E!vv1&oc*mn)Q_mZMZ51(tQp;gk&8=iwVjhxT8HQAd9CR3 zP6H-`&e_3X7Pp(AHxI+D_Ka0)K%vm|&dyHm$buHjfQbU?dg4x#RY2u@KXLx3wP5%Y zL*_4jJFAs;^5J!7{icH;s%UQWayvU<%)AFe;&);;H#arDOL(505^8*m_4UOOfei(D z&#qN-CPVUGOf@Mn!?-fAV%CE6CqMU^#e*P%u$?@a>48R+wp^M3!enLu1_J%nN}Gwf zLziwTsVAK|GnjQeH^6|-Fx{)xl9m%2$h@2dWEFR~Q~ZDlI2dSRJG9e2?aX%K(l@I1 zq)k=*eq?$o{ri>(&3`o1_v2WmKfdB3=#jadT@g!6C&OVw1qU!^|A%amA{tL9++W$Y zX)C@l(=NtB)kR~#-03@QGX*FPc!8%jlrWX?F_=_dl7y{INldgN*ap2;>-D?-2o(5E z-^^^Fj>})EjVAnFHEdzwkC5p*P3G)wkYmS>^T3L*_SgWOBupJweM3bS5oOubagB!DaF{7 zw~w?fM3-A^LeBsZ-r3plzUf!zgJQhr zJoS*dzx)^At`A^2K4-71rNc5KBaV`P0Yn(Ct3!;n7f~bO?T?@h1b%$f{lY$ zG{=qaCXK(DdQj`4I^8@usPgw#P}lS6*(ZjIwrw40a`3b9<@;Ttt+IT>goEWz?y7|9 zg>3byTT8~iHLa51*?>NKsAbkXr1boX*AA zUF^YC#(|QgIy+<3rSD(}4cLKuz3KOL4b2B>G`$)E%p-7H*E{#0N@#F9BJ9lkWCiY)uX@3MaAp3U-e1g@ty(Ub&WwgD7eA>nTUU0XTLohXSn%~+i_6pQ*Zq-eG zl5vFHGZJ34oS{)&tI#mwo>Z5BfZV^=zaE5eX1&qO-Jbo{1qw{*j~%4X~9c$sK5c6??6_nYC0MN3BGx zd1K?A#+*;L=Cc^sjs8qIB*JXQ5e~AxH^_pUhfR#}LiDJDL){d^w1o4v^@{BZOocnQ zA8RDaHRW6ovf)!uymtNNkmUaU`iQ-&s@-&{OBv^MmY%OE_^`uXGo-BP#jS)jV&r;D zktw&MSGA8rN7WGDlcS0s#%gtArR;TLrmw*8iblJA{#~Z)Q^l~;}3TDN0T}Mmb%mc6@@CJrlW1uOAA|;+rD0ujgqCwdC(wsY`RCB zHA{Pnz)Zz8>P(xuU#6OvY2<8mL6KIEXjPCx)4_*n&D(ka7W|l^6pbiGD_NQ ze)1M@OkXUnkc+QXUkrYriS<59s3p^;r=XkcO0>F;306JS47LqH>Zt>q#W^`+{K3IB ziYy*{1AdRO-DOD)!$^b}xoc^ydTdz)b-8Y$v~K#BrRia#TY&GB53IS0RD<)~aS%O= zL093;ArCaU2rg($qho1&OfXT;(`Xv-rR0(B;X|W06T|P4*S68J9w?G{hClW*rJBi; zGdp`Nq(vD6YL)a$L|k7z<>x?z|1zVl`PTQpmzs?5rW}WR0pZ$>b)o3ZVS=m2@FBg= zt&z86=f4BKl;s%_$x6DaZPNS4SI=K;h}y%ra_nk_ zOtIb}6>`K?U?AvOyo4?vwo@XY#u(O+w4Adr-By*3BbZxDqF!tE?C9p#1Rcm3^dBis zUu7tkwDW6N8Lti?uaNwSZEn*v^g9{{lISYzD{vwl*SdC%E7&?o|Ioo2UasM;o=K_p zO2R{v+c8>N$enwsaS3c!<6k2)8ek>`?A0(o+*CD{d*T;w*oN?-ess#mn7+F*QLXAQ!& z#x(x&ULD7unf2>a>MtMPEgxXdrqI&j7@5PZ8>T!PTLMQecuEYtbHx>M&3GR5=wQpF zOx2-w_*G_V56`e%i=!)RXm_e8Z(YQ|%vSzM+;LV~tT!ri<$GO9KJ<_nf)l<oO149Cm#QdIg&(g(tSU<=a7dx-M)zR z92uwPZhcgufG~q*V)p|gNJaB8jGN7Juk=dcHERdxYERCrR2n_%YPIS+rBVAFqJ^6N zvZOnV3D@LV%V(wGE-1_*Pl@861s46CHQb)DglCAF1_IxCQeN_qY>qzY9y!!aF{nrj z(U}bG;g4@u=Z6(DkesH|bL~n__Z6AXm{wcCZ-!&v*tU7Zb1ym(Rt!@#eUCfccygR_ z)+LM1zTdzl#7mB?p7!6w@AJ(R6?2W2*wu%IK9nn*w%=PMg^l2PZen#Rbo&CB>8hj8 zXuMt%9Og4@KNgBs(5g)tPQxOfjSZnz;@I;Ov;|EEFsH|Gdi1yY<6B^nVt&4(9A?>l zucW$@Gfz8j$plQV-9F|kriJUT#NGQM{De!*Vw){DSYVYp%N!?tZT1qb{pn{OBR=UT zp2MBo-4=RN?)OU)pI026Rl7$9Z45r8Vqn~QTZTK;zhL{#O*_z*iO{21w*1q;$au@l zev2#R9_o?C_2xV1R9mlwS4QbKXj2sJ+a|p}y{OA8!z#?LrK7z{NVK9~?tJq*$5NVU zfkZu)$hbcBzmbN5A!cf+W!Ei_JJoY~Hew0KME-)mx|bS?=70F%R}|@&UVq>C_B8Wu zQb>B3B-%=FTjLa2Trn)=f5-JlV!r!u28}GKB3NWf8d)W+^IC7daQA=^*(;} z$j`3iGFp8^>+z<+t5mRlIX^_BpbUt5uaNGmB!4Qu1We_^ia%hO65PNs)Y|$bR({@y zg`e;kJ&Dr?+oP&RM;o3aoqAf5cKEZFgzCqN0~>9k+q#c~xqFCl#4XTa-qygqdKrsP z%kCR+StX_cN87*J+xLW8M?CAa)LoAF}93dG36lqf?$QkzObovpgWp z)!ryZjBy=E^hsw^(1Z16pm(p9mg$KXAr zY&fH32`H^f z*R(E_8eU4QWCl*-IvBVY&s+pLsgJWom1&UT^iJi{Se*>-kY-uiNtGWT+&1!;5F@%V zEuQ0i1K842)`87d@39--@V7VOcOLvCvA z(S1+*>C}X32C)vT+dFf3G-3vcO!JobD*sk1omfWgFW zvGjX84^Lc*tN*w}%Dd|X2Q@<1j|#SbJ=6Z}PBq|r+~L!bD6j+iVYs7T^e{2eL|_3x ze~(Q{KiuR*Oud*14xc<)QK;3NtNRr&6=?+R6)f0R;P0w+Qw_C1_8Q-07Sgn~ZdfS# zBR$W;*e=**1*rC4cZKQqhKU7Nt}bI4?1(=+qz(VBmK$bbd-&TWdTojD(maXV?Sjv3 z!qB^s%IpamqCKJAN!;pQ(2Eu5ICJe3cYM4-+6S~>deF99 zd{>>W06j%pr5j|y$X8C?hj7#?Xu^6c4O)<%l5#k_hIO$iUcibslyd|kc>0G(BC1hW z7yV^2j!OaaqBYvlVEvA+Zo0a4U8)&rsGeTkGK|y;952~YK7~jc`nDu`-9d-VO5$d} zS{Kj!%P$Mq%o}3Y=_FpW^T!P9PQ4RE_RbCt2Zr;hbCCRgM{=AjiH1^!m^D8dq9kzK zR&nQ;fhw#;($D_LcPRmaavJg|-LkR#T`M}s(_?r%{DWiYc1%@l?ShSuD!L_fc$XAy z?et7O}SBP<6(RynBIMQVD zU5(PJkbP5lO)@%K;IhgZheWpsJ$&--Cp1Ss6k4gMiNzvqgudMkdmZ(ujl{eHm^QKh9M z{O}nGo;)hXy+^#rTKunGoDHf#EfOOA-D|I5y9VEW!*2ketUvnO&(jc#Z{>Cf&@$t1~s%^d0?C-71_uQYmga;)4?y-{T zpUINiE|_07PN`m8t;uGPleljJ>iarNM2>&+qgN0_Xf#nL`2uOHCBOi{u`t(LFrFbj zTI3|;BQVKo?b}x?1i8s|7P20c`N=db?r_SU!S{Y+$QQT4&uDVfhlamxvBCCBJFbhVO)-s)Ma8Ws(Ih&53`2l6>P$Us6J6{8BGnD?aQAkO27m; z+XXKSz8Sl3xZ+!ogcy0}BedT7VF#N3v(rOW5(GHx271MyU-JiYO^mg}N;i`NGWt$bK=Rj@14kj$0bx73}wa_c6g6a%w4T)ZJp!UodO`|q zuiQIPt0q~Zj2@b|`jF>R9G+G)Wt2{Svqbz3urr*WYbm)<1~noGWG#VkEs zD98-G^WD@JYX<)#*)CWc@s@4T(mnx`j>4m>rxu=RcjXFx+JtKDsO_mqI9DXyLGe~# z4{?~4i<`y*7>Kb=g)zpdzl5Yqsnmdu3lHFxLPjGGh5-9GYP{q}e~??q-SBk8Tt0X` zWhx^S+PD{!eaG7iV7mMxKc1^alPO`csM)L3pX4#k_OQbqTh{lJWlnfp!b`b*Xa-KG zM@-iF4;AuIv1zpR-C|ao=sT2VMRCE>tEV~!0)nK3P9M;7B~ztj8SaEZ&xS56o}~~z zJjTaZH5g;w|MeprepF}H%EP}yBd80b($IG#75i+h#(u$DUyDgjKu_}8f``41bYO2}rKh9(AVse#N z?@<$aU@!9l?WG~<3cc1lW{0ze*;=w)BHs?X;8{fbabM->_sl-!vxgV$j^WkW2n51O zK)scBp=3aR`Dddb9kOun!c*1jB|nrKR{L>;2GM4>N|yHVhLXX1bbe`mBpY&0-uj&P z!d-Zb`s{}y_nxBO-00aic+*kdoHVAg6hAwE-`lY@qK!GG+c9rlS__T*#mDMS!Vi?w zIW~j{L|5fTaoY{;+DgNpUru39#*BFDgbN${S$YXo1+jzX0Q=hhGo@BONF1b1SGAW3 z%k!UyLSpjDjBZ3HR$w@g#YVE_dE42)lB;FxO3LvFlw>y|baE*3*YWb(4=4NURF?lb z-HtnJI-YHCoR{THRyd)35005Iz|qIi@VkN#`-CF^-qse+H9B`lHraveY#H=Ah~QUz zuQrkcuUITwzL-AstN$u0<9hZb`?(Z)prx8XUP`X^g+))~rrlV;*YlrbT}V@(K7}Xs za@yPdjB+(;5ZpUUN8@LYlxh;(*oHXF?%nlAHh$?3b1eP-cVPKUyTUN!<@%y*X7W8z zap`lLsXoYjE8%z`?Qd`0n#6sp`8eh##nbXt!FX&p{5!yaWS7%A*#Fmn*d~dGw@Hl% ze1HgyoaDOZ@c0c_s{ufaiP<9$NA>)?WQhw?ixYOLEX`=g35!J+*xQL9db@e(w@a!Z z3KySW>-pV%EADneof8x5mFxN3C}=v)H?2;O4?AaYmWF9?UnJRK^~$4^^)*Rw<_cvK zcC+IsJATIscWHxuaPM6RfKk?7q5-*&mJf`TBLWOw9Rxj{@8XEja92c@FT)Z1rh<7_5V!W5(;^I&_EMSp4ZdkKxWD-z}}0&(HkjWQ`5D z>S)7wGj9y&n6e=HypkeYB za`@Tdb8N#FeD#yL)|wtu(;O<=4$_Q~NB5MRJqFsd#h>kK&?s37YR+E-1TXycR`yJ= zqlc4h)~h!qW#?@bpkavwj;8k9h;{kfDgjR%)4yWXjM%GI$xRyCrYzDTxPR0Z%0UEQ7b8-tc&-T5c}IbLYck_ww7B9um{P zKhDlHNFQ@a6%hyrzRgJ~d8G$+Tkiz#`XHJY`nXs6wd}ls0R5(TYHbR{W`gEnqLIp+ zrohiPzS#XX$=p+%Ph-93qQzs zy|07$rAj`aJp{H*&*m~;7+JiS8--#vIk*5uDb#+Fy1Y_`E;KJM&k9{gZ67Z4=yW;g zpKb;=zE3$eUE?{GLE$WK^1y(%{gWR5t>Cw8g_0??s4K2C5mrk37C%x#AoM$en7{nZ z01IMo_vpKSvoWU3j_50xv4E(Rbd!?YogYFGdlNic>x^y05zjxRqDhCY_8%>B3bJ^5 z{Y#Idq^xb??88->%Kg){Tg+pZ$+p~%XOJ&QcY)2phG4lQU2$3(m$v0 zqOQBni~=Zs7nvf~3ntW-*pKL9{!Sn^S!hrLI*4$)9}ZO_yD(D>eL{e-D*Nq(DVUI9 zEzasU4wGV(BjP#+s<5-nL}-!zagFIJrz_f1U#A_gIFN2e99FQ98Q;6Yb0o}M-L{{g zA6|Xr&5@tI-s4Y83se$*t94@8_)cO{TMK3H1pltDDb#BPTYq_v8SuO5nZMbcmLvqk7+c4Ib|s}J zhZS0CGIy!B&1jJNvMoxXVLQYfEo6Ci>LS;6jGsT#4qHXa^CT;2=Z##HsCh&I-1ZUH65)9W!J}-8uZFn=KMwc1DxWbL3gm+6t2vwxiR@SP-#n^%1P&V13feq!T)+0>oS`OG&Y} z83`z{Ly?D=ZdAAbQmg_1kjf1g3-{|eO3RfwaN6?cpuR})^C0ev(;e21XC+Q!_zdUrLAd#3oh%Da)&HMH6KO%b{MD=#^%1l2s+5Hr`nND84*j4tp_W-rs zA@YQ_*}Ji-A6-4x>+<;e{NBAxsd^xEt=vD32$iVyka}5mcFk>Ja_thOaww&jt9B=wo zD`~TnKjfJ>1I1QoAKJMf+0+h*34(5{We$EnX5DP8AG zH!=ijcVyW>Wp5foeuU+(lTZ_MG!63?a&-PY#iZYj`-;{Z9^am7?~rtQ(GEW{o-e6I zz55u=!KfAGYMe&1+(CS8nX|0ucMcIE1{bv@PL~ZW-#kbjU|3Dd%7hlETer4w;JxEF z`!?^6sivT=dkNVdxrh zH3ef5WQ(UZVwjE;3?c35lHg7Wg_i`(*K+4=@_ zSi(Ynu~AUq;P~TC9Z!6_U^xeu-Q)kXt$7_FwOwQxF$0-q(2v19f{ij1@4|prIGtXF zbuYg^ZVxXiq6PdK!ZfxI>%{^pqc1ze?ZP|V!7{W@r<@f5D*VpCWhxm0^?)ByA?>4=z%^o^z?R4v(sE_} zVzylHMQzj5D<_WwnSeTZ<%I7({KB3gW|smY@0bY9@?LLcTK=SO)B0zyA(EhwRoSoo*BMT-d5YLnF3C^qoAa^Wu8#Ym8y#l0- zZS$S4hwR*|liZ!kq4JvTw=K;E+i3m!Xgw#U@s>Yg*>>hyl>*)^& z0tp%J#PHG)yR+BLJR)G%ZwBwd1O57N)D@M`sl-12RX_Y+%`C*0E2r$vH8F%l@}8KVivyuiAh#Xig4!XLGw5 z8-VrWd#pV_uz|dhEok$Av5Ct%FNa1IG=c{5~qrf%{Gk3=KEBtiJDu@V%#P z7wqTmwhi24!2E%6hYgv%SsqLvF)A8Vcj>h~PrHySE)mGT7x$W3 z0j(|kLt^6KuV!WsF^%f(>{PM_+wx{pfO`~HUaY4U9H;&d#be&|PwJjy^rXAG2Ic2HjaVlD5R6BwMb zGl(=8YVh5(8pK6W`aK-|GOr4|%49w}z3C5%IyG?TQVht4fL zReIFYI*gV14`8@63=CaAg*n%B(HVpq#oX0#i0=!vwHPapCjOTJe^4dkLk^|jqkeAY z-rY!iyD+o4O=p7;Uv2}9RsFw^o-#RHCvTR?ojlpMniTw*z~+>(!i4L@ggy{Ughz<RRmcVTbfc}~s zr_LG7%I0I|saK*fd~W;H%(bee!QaDPKC)vsvdCelKtU0-rYTESvhfq}7uBN!|?*{+l+&J6=mo><rrTLch0LvJ!ewthJ7P=qWQA&fqUfq^6dY(U=Zz;uTwjvb35MC0`FVg5mpWUwG zPJ^Z*N6b4H_!1v4WCV`8_+8+otI=Q23PGH+b?z_Wl-t8(DF%3jm4Mq*zbn_*jD6`& zhpM`lOmk`gI<&31=kF|VyDL*o)O7ZZ0j2a!Mpo6Q(DQa^HXjmGbsE4%B|QOL97+8~ z@3>d>)0?~dy=lIq*RlwoDBa7&&5y9WC9_t~BLO!a==(fAM9m~thMZh}d@^pJFSBhiT2eOn&Ty+l8U0I%Udo{l!b{zYd#b}FE}LUq~R)YLuK)U7o0Sbw9o;8>D9iFthnu{A|SCe zx)8|p-7NnQ^-M!ky+MNF3%6*JmXhScxuU;QmpHYY6`VZZfS+8MGdbKal1Rb-Ur(O=U{eVd#yX6Gl#8W`idJ~NR?kpG~)dp_>9czoNOBsr-Z*sniT>=M5 zxcf;6v!7Sy837c6oN-#atTEn;`wS?FYKcH@8xr=n$0j0is@!Kgog+l!{UqM#A$~5= zB!$kNBLuF{G7)}LQ12}8#&gCIgmXCfkMyb%n{sOaw=EftlWpiT{ z%*3rn0ClG?%?zvj%$ghIs~fMQ+q)}OE`5Y`9XM(O2mR)zTHC&pFv6YtF~v4Y`*Os`||YR&EUM3PZtf?y_p6mH|g>;9vd8IQelyiJ6&5qy@S5 zQxbNxNDE4BOOyR8t!Or7*TvF_5wMNg2;y|FuDfz1elqzb&tPQ4JIz|(W`BN3kgSPX zH~p7eU?KGkY2N@RlD>ZQ*Meg|FI1*8@4Qhsw^G|8LfV!w``77um)r-I3_-wQaBkTwZ;EcBYQ4LR)jn-;;xW3t&*$8^ zfb+P(jCo<{w5tB@s(p9^eXex}DGM!=ebjX8Vqa`Go8MNCk3U9g!8ah~W3Jp;cY5BJ z-u~OlBUcu$Tlig(-L>5AO}9MRWr1&n4!`A>_JA#+k1~iof>_S{2a*j@x|fw2-g1V7 z2$&8NmS|EBg;UmVG53924)k}|bd55i-?tG8UGDG)vKes+@7^UaRs-}XM;z%n9PdC$ znJ8=OprkMKM9hizY;A7Y8Qv~^nTqOe^z}>6Z03)x$Op+C6q5BW4k_KX`Pi_xFxV`) z^O{D%*FQF~1g=Qhum_?%Qc<^m&aS^?$n?A1JZ6W)B`2OWb^LIdG53?Iy^!<@4vyHKM`u zdt-^;;rkiT5z~Oxk&67xW6Z!&CY&rKpn)aVLWds-I(|=uFHP+i$w`_urDu{mRM#=G zXTyIEx|%E1j*Aw2?g#wyHdV3rHjXRjy%26=8g(G2=U^lAWu*zZRJEVGfNMO%XP$il zj|h_bd#MlEe^(x}$r|}j*o7F#JRi80{VR#pAC%QFPw4k%(|E*(y65M4uvwddTq$`H z8K7ASSFP-SZ~J<#U!DJE&PTF|l9HT%Vd0{(`3>1Utl6st^4~=8rpo%9bJN=`6nRY~ zb>{3Ci%dz9`&~Kpq__K@uZyCH*}$(*Uk+A#wjSxeIt|j0{dy=2O_)A9o5`WsMO0PE zIAqBAd?@L|jl{Gj&ErDy&RoyDsV)S&DmAbt%H&AzFOW{If#->TY{V3TZ)E=#N5@tW zad~IAW%q~`c>^*src7Z|?5x3Sk`IzAFpCN-SGwP2zGYi8VvP&Edz7R0O(EDd`irYE zR(4lyx`fAjb;VVC2Z;e^&fF1!=kP=-cFLh%37fbRmZQ5*umZhGONlU>{f5s|oUFm` zQm1CpB)uz^^Vc4KFs%hTKjx7B9 zo3bp)&{CGVJ8SBln5fIH1>2Qw5Tl;<>B2$nUtM!7@c`WIw0GcXRv9n5dpEW0Z_cU6 zL!m2jKDD3BA_4CB=O?N1J>XVlJaRHRFHfXamu#(99(8hiEGou zBP&+j+`I`YB&HY!t=^S^oG$+XJ?U3nVcSF!Sp-^kjge|<5iT@b&e-h~%nMe3+1(5% zQ7i4Vt7ay6;PZCN_VBk&&)sZ0FZyWfg@fyfJN2eZjBQKdZHr4t?}N`G-5RGS*Jh$u z+D}ir&6;=pM=Q@N5(5@G2oF9|CVgU!?$mC%?jGL~Vs$^%s{KlL(q;KzVew-jx=&V5 zQ)N#6<@O45ay>YyD59hs60s=I{b~?zlfi40a$Ra%FJ=!y*oRWnM!Ia7iv!8gp!ro+ ze_Qq?#z8k5R5*1Z!s_|lRXy%C@}RSGzucC-nM@aXmU|Y~wR9u;E>HKrJ4W#*YKUMu zyu7#6^-w7D?C+R&jIhOdi1lZI>!CY9sbKSW2loA$W)vWF$q4rytQNzC}PnLdl zv0eJVUThbiqQ|X-9h_=B^>>aPS#LhT2rb^Ve)?4>W>2yK{eP79o?%UGUDt3B6+ARO z9zYR9MFFK3r3eHYh%^xdr59-;T}o&{QBbLZ^rq5PK#DXWp^Eep0jZ%!YJfmO5|X?t zc#7Y1-{<+>>w5D;cM`Jp+H0bu<*!NfoR)Q#1$0_-8;0G-!*q-F(# zrj+_o@*CAB6JWScCat7JsR`!bAoz*8gQOMsi6Z?ccxxM#<+F~2#WpscVT8sDiKX+pRKMf!fbsy|&ks9eHBUfIOBy=>WK51~T z?ctmJQavM(IZ=%>P5S>b}{xgJI;=SW*DyN}d`0Yi>$M{uqjUtiPp+EK}Y~{l=e(EoMr& zF2A3f-maOTT~*~T@J02hqORWDc~feU!u$fhncJb>&EL8i@PPwLSm!=0o9xC~;pyMV z;J#)xc%T)+K8WR-d+acG-X*UDzP35I)5HF4G!y1dY4n#u>jX^?SyV~Vh60H%gM201 zb>R=4g3`zO_cuy?mku(*ESvBKAgCfyn^<%=Q>Od^dmM)f@wh7!e~NgR4ShD2cw!?* zYxHa(`1*#09*PPy$`QBfLVf~CG zetzKn&l>vIvKQJ^`Le(I>8mI~NYWL>%X=Y5tWJt4y-dv1?)R_KWh&T)c{2;OB0H8?M#FzZv#+*s(otH z`!HIC^g~T4L*Cu`B!mAcUK?ZjLjUf~B`|%1zBM0ChZispU;?tx?w53P(rxxH3OVPu z`hZXnyDG4qK^;9-0|_3y2pom21Hoz@5iXU?Y<>rVa~ID*Ao8cxvsEBnb4S=9^;Z3d zPJt5tbpuqdAHA9$S_gu^xPw9j4uDToW+EuXFb7xqH~ClT6gi(7K4L$tAj&S*KY$k(cGNp;tQ%v+%?zbuv~ER zF}phJIkIx}Jcisw<^8^LAhrW0kZhlzF>iI8vz16T$e-`;>+%dx%&<$s62?-(v2tY3 z6IntAu^leA-lQC+%x(afn0FNOvkW!^=dsu%eMGMM6RT;LjFppZ&Wcv? z6U^PKI3!9dmnCaiT)4KL(9oksz{(yMqeQYNA5G+WKOE7X$75d=#|yzo;bf zX+R)imGoHEuLCPL)x}s7A{z_HBEFY@m;38rrH`k7T8;iL*sKa{d0G`dp3svCg)g&16FqyKVAYevLbsRTKT&q+myfRiN`Z7o0k-V@%6x^`yg%{RB*5WsQ2mT$?4ll6qp zFfbmvRt|v9!O2;rlVXM4UexS|`t`&+7OQM57@>4`a2RqAlE#;qM}UuVE+i*j=ahQQ5p$oc-xyk;c#;oE85+*tI}oKVyh~ zo=#ylm+;TMSLh>PjMtX-{z@Pv_;zbOm4%4UuR3{4przRcDo{n51BZ(U< zqg3A#M6RQaWz03zkMssSU)Aym^pc@NZ95LJnkk(Ge>{kgtrbB1#3;SRN*3e^xCyU- z=bXcLCHev+Makn2l&YI!)1=I-V-2IWJ|AkG3wRKPYk?D%Buss=0-3N+7MFh0Tj0S| zQM@b=-!dD8f@q&MtMghfs`Hj#02}-FGHE-zPm0)e=YQe8wfuRquSOek$V$iCma z#a+QD|Lq1&840%LRD4H@7PYIj2zQJ8M8KEUeG)N|Ynn(I#Uz!hb=w%QKS;#UCn`78 z;lA5$EpTrh&BorHC$rUG@*|i?Xk%%kMw^+fQbg=dW7((H`TRR`lB8WgEb>WjVyi-v z9IXcn0Nzbb=%T2oXxa^bf@;dgt*xyIz?Y}v-{|jW9uk_TklC(u1vs3jzL!7@V*F^L zC*@}ZlfaXMC@m$f+~NLJPym!G-?gj<-g$bC4n}L-bbJY;YDVv-y(<8s2(RoAM`MqDVt9+o;UeN0w@{Rs8koh~;OwG5R}3~JhbvS0ty&(`il1S}1Qy|4+` z8Y~@i+L3>|&$vjmF6)D;?Lcsqw!rM{q@|VB)bnqTN9nw0)R{*O*8-4M0cyHQkFSg4 z9tqFRZp>TwdM|q!w3bKDN*B!Fl4og!>GlT6%4_Pf8@tMi8WK$Qhp^8}#clULz`>%6XjZmN$cZR8} z;BFure37IHr|U)@>dPePO%j9nMFAWKF#gf3$LBa9-H8nR z0^k+Rbq-&6PWjDsr-bIG&~*sy`ucf$)I{TD>-#~}!xDKu$d%HdeP=@#18J3Jef-My zLVE5mY$y6%t-NO&e<--8(Z1&S9EUNnKgJR^!Lm7Oovorkw2}@zlXy}naV9^?k-yD0 zX=t`Ie^R3Pm7rzpQi?hP&D0^q>C4=O{J=owtl*ECQH6aF{;NFGt^T zv&UlPOdm+y%Dl1B^y;cdvdDFXYF_msc@1$#X*71hJz{@~;5l_PF1*KVo5+ZEx20Vu zOHY7h^In$od2%cw<_c;%sHugu+Bf*mPD)${ObT32q^#!T!}mc8J{6TG$bY&XUkJx4 z4)gGyXggEZNjPSSxGGZi11<>u<*!@Vc!^U#krpz&=EY2d{b%w=7B&L^XB9)weoR;yT7IrOn0);wby&?_fomM%)AQnVq1U7q`;8)ZF}W z5=yCGI9svfo&1<*NtUJvwA=Q)qU^I-4>=Eyqrl!UMEcb;i(Nx5x$(eKUS8*8JC5uT zye@t;3@ayTZ+IA@{VBzv@F@uRmcwyfz=zAduy-7gjy-a}doK0B7b9hqYu{iA0jYQb z`2?5;e0i~$)D70M2ZKoTvTnKT9c_s0!2Lat;42>4ReoOndtSf%=~$@g0_5@UT9L+B z(06~__!YO#0mfZ7Uf1Y%NaXa9ahLFac(h;OkVYn5fOA(Cp#I2(TZ)O6YG7bBF#`tg z%f}e5jO(2RkzuF^93{iPj7G78v!9lYF_i`=k72%v`1I*09}c6t)72v(I-YEo2A(S= zaTiU3CCnp3V?Nn77UmE$+TnP=n=<>UZtH2wu@^Kl#~Hf`yM2B(V$}KdN?kgtMDR8e z{1*W`in*=`d1)%{@<0g@)8E%T_AI22ly(}2K;!Jq5$IknA)T~4+DxxSP0=Zq`-wp^ zVoY8Uc8UOs-^+L&`zPL{+vz`F^(}xRE)lZ10g;WOBf?$CSW?ob@e?6}a3UlH3>Zjo z)5Bl9uogN%ym^H#X7E4N60V5vffV|Hz{JzNsscZQ&;NPXzdj*vd=-d(X#MNWeHL;O zRqm>!FW#BdRq-hIZ2QufY%SCJfCg3s3ybrXVZqxe`jUo;+JTz9nZdep8G<7qgbM8#rAd_zsiV) zjEjuq8YM%Bg=Hb3;{}ok6{q44sL7v7wg=fjozlRy%hQzOS{-$Z$W&LCs21&BU+_(_ zRZmzI(QyOj$NaF>^jxFZS#Y5kVmf<(;$Ioi_r#s;c7CoffE1QQ0X!Ul#0)OOk;9pc z2iQ&}>z>oc2@B!M#A0D+EbLVr3}TmwQvyRj^(DO%b@s^oECaC17h`&Zs|A0qRS-!O zYc(^Ze4??j(RjD_gLoVSyuXMh>L|k*WncZ@;`#s3Q#Ho+sq>pfon`2i2i~{$X$Bx# zn7rYBo}pJ3M4c*xe>zpb*U&Eq(WAc{5jFY<@tlo)j}#ZHTkSdk`SM1W5fZ%r^D-Ou zx-02DO>|{|C6ku3xrwEQP;o;()S#MqCo~#PwMTCG^mkZnH{ThBt+e#y;3TM*{I-VC z+7`Svk3%;tSEmbhh<>2A#kFk`{VFR3HoJqoiO5Fe(EQ{@)Zx zgM>T*qrgBK@gvOq{VUG5%OBll7t8fb_rh{+B^>?3_BHeV(w0+}Rpr82wW}ReL)u_y z{tqBhrW@|OYQS~W)W{q?XF*y$)x14VCB04K1Y)InhL0)RA)d5zbsHm|+;vn^2clHC zI6!}W`}V?jqa7rPp@D{3cXZv~x#q7_R9z11CXqY1_|_`P3EFBJQ*%2I7nl-7`g56!!*GNQF>eT19)% zZ38@d3r^0nt)9c?LgDLW3o^7Rt(96@k?{L(@kVc#H{}o{+PH=K6OK!6k_pKmL=ck9 z&IY15Ds4)=cDvIX?=yQiHjg~_QY2jZ`dAI@Hn@F}ud~G4xmnybm{7Y+Nw;PZl@zq3 zl@uzRsk8y#NnDt(q~P8Lc-*deBRs^)!WR16qq`}SG)h^UVlxVGMn2eV?V;Myyh1&9 zi4nS9O-!d;;=SWZNgc@L@s_6%K7>s5;nVFm{N0Q0;5RlNUpLq$@bElYR1xLkcCbu& z`vLb2MLQR$4HP3;j_G|lYDhgQ=rRdITrIdi`1X)*4MVkG{-W5@TQd)=jMikEyB$JBw^gwkP#y`Aq`n(v%S zM&c!CS=92Xa22-w*Q7IDH9?3CdK$-@oz@wa)`>OLnJ)u^Iu3b@D=F$P!cY3YbRB2a zQ}J-x!}P|ufq$?KCr_PZp)EGzDoy?MAKyJg?=HsybMc@6PkFd-yU9dJ4JPC>o0b<8 z-cd{}*|bI14%hmFB!HcT7?oVh{-XB=YXq566u0@9v3K#@DF?PLg_kz+?IQMJJ zE3g6`ui}d}u*Rr+6Ar&YRt{I*h!QY1jJj`jl%hr<(x}Qd#To`|Z+ z9uYN}9X*%D8%NfE@f+{DCBsKOaB;s1EcUxi^=>!K!|m z-dB$uJ2u|@=*5ko3hKye$4U3~nFTb3$ua8=zJ2H?;S`bY!#ry=6U~H#coQ-U+fnGz8IkAx@w{p8Y zA)_&~Cx^<^gFq zQF9M%sPD6br~HeAx$AbG>YjFZ{9c^W&cdy5`N=-l+Ks}6LW`O-apltCM!7v9>eOhO zbi)ULx?|+en&UoU6FX%pr|%tm^au?7P$P5L9I3haYITnBo8lJ*nvQt&o5f|H9?M)?md z=FqlZ20^uZ?-i%P#RJ#DvIh-Xt?cLjfQ%FSEZpKB)S}VF9=TOA$fD3 zK7^>yUdlZ{AzO>IYpKIZzUSvHOu=_a|B>1impq8)CW1_$c z40Z$l1&G1_!Na&8S<&LHuOJS*z-hi13xSMawTM+I5C@*0(Z|0a7~q1h{{_z<-eJE7 z7Dg1DaX@xY-eE{2NQraVwW|y8=;MFF^wC3qQTV@9<(Esb zYX28h7ZZd8<4wecASw0=0OSw8{hRaG>g%^u&;ZrC5552af$L)#^kV+2rrOf=JrIX# zK*`Pgr2Od}rq@pn1OfhpxGR5$kbbGkFPE>`{#WmXukCpcz{lN7;Gw6FuzjW*)}?s{ zxSt^3*c0R%caun@UJ>(aQ{F3oCLDkH?+M2sZDLeS6kdBLNQ?~~wCpd$HgqnkZ61*nN&{n6y(9}q}%Pmz3Pni#$$HPmUDkh?&`;Bj>tY&59c9F|S2 zf=R>H9=BK`GC%&OYt6T zZb{{`uj9c4p=uH-+&X4n8`M|m*?zv>%J@dFG|<%s>s)1;y!%_icbn)yX$FXFeSZBU z;62|vIXUT$`OZY@iiwJj-3rvN-Y+i-)GKd$U!Tt+PFzc;I`%U`WE#Ts zPB+W2GGWZq8^k&dpT!G#@G9^#qwy@iVQ_J~)F{+Jm88 zCPqbPGidX}W%m3!ki+x%y*#!chdwGLQ7E>P!>QjW?H*PO0y%uk^i>ry>9}b75e`n< z=$V;UHZNehxAW5HB|6Sma^Z?NM-|PK;I4a;9t#}c$(ElAd?3*lu!W(AF<4XObUwDc zdZ$eSM+T%rAcEP#PH7^?6Hu=>M`8KCzQPPzFM$D{-$VHR_#>X=)SU^Xsk%#>jyfi% z$OhSd+p%?X6E{A8^}3!5h&%-|TxR6C2JF+BRbwK|kfKVb1j0m(FAE0m|DC7T4>Zi~ zE10o72K@CEFObEaR!9v1T4sE@5VlD?AQI&7(J|BBUJxEL)&)gDB>8^5qW=MyxQ2x2 zC4zwTNt3&G7tZ`k@q}?7R|B{TY$Mx4v6uPJsIv;2gEC!Py#67bznNUy`dt;c;@~PK zT?y9ja8p$PD1_~+qjMVIRo~ES#S50j()jrJ)%qTw7s9YRg@ru;BSU_a-J%e8(-HqJ zACS%8>NQ2c4WH~U&j5oHVyXD+75#@GV9Hj_EW0cJ$UfZO-hO|N!-+Qxg+hO=A@Zs$ zAl5E#0D5cB@L~_p?i*U)+Y504JX!FS3c&1WL)cFLQt5vlWcLq1uqkl2#6K-y{x8M* zpaThNTNuFQp37s5{Fp?4y!$^;!Gm#hjZH23X($kO3ke3|XQz+%GXK*P!X0T*T>5zdT64Ha4@Ktp za~0j$^LI5;$=(;F&I1wGJkWmiSGC^eGz7Mk1F<+R)D=+J_rd|tv*MW9Y8*YkUVH>J zI+q(jU0ij;mK+q)!G0?#<-3SH_`W0?G8tlJlhS#L`8?~ z{_%tgybS#K_{Nql7m9MeqZt$EDFsA(3b*;DC%H0L{88JPKItyi6oaCrv3m0QT|SW^ zubPdEv(6mtmvko34Rc{#{Ah*)au7+mUwYZXq^M9~Sefv9Z(Dh9X#JgWPyoWAoJqw#RfiKFk<&d8D^jWQb=%>zbqs#&os_ z`Qn~-mJ{&?=>ExycIl4zp+wY=gaabyM;Ef|0w=tpatnLz*-qBYwUZR%i8$|7UTGRC zo_H~y$b=fz|2$>{@+IAOM{3@~k14%=RcAf!!W9?-$wxh;&cWQD!oG1%bOv}&Ot0NExEv&IsmlzMrtnyI@`7zm4VH6w<*gHstd0lDwfE*GhZi=P%Yu? zwc}e`+PS{*X|CM!>!J9$feoUIv0jAD8V+%OWd~-|f|l`V^gQPbb)P+$GjiX`jk@?T z)eA@HA4Jb09ubZ1&+}`Z#tochfJheU@F2I`US-bg(KXFij9Zl06ivjA-_n64&&e8l zRj)pqQ*w&H1Xlwsq zJU%|fOEi!2Nml-XHS$9LHtji<|9iB)KS|#(bYf<)2hVpHgYg*ussHDo8%;@OvJ?0Tw#40sZ8m2>`k>TG~x&k-=T|lVEmdh zcQs4`MaPmW@LSt4M4+o#u6U=T%F}pr3D{ zUA4~0IZbFxMf--7veZ3`?(D*ZmZ982!^VNZuhq!Hryj;4=+AXK)y7ma@x->dF()Vd zN$kw)oe}>I^m}6tl3O*uF6(fQj3cJq_tUbO^VY#=1>U2Bd$tWmCiZN1^UXu!LMx9P zUp^eEz-bZF@bJUCxQf^hR)!q}Y%$FF%u4rtp1^pL!#mgE7%!Ba=bJk!)TS@+&6+_7 zA#dD*%|gQ@^R^CEjV~q67arr&finR9np3epLz0&@UjxiiOaaHF6v{MjPjz&Vx?` zHMeXxl(0<&h=~n1Tk^~u4HL1@M3mw8kNV+LNmxX=C(Gm*&7|B;KE8tFkFxd+U9tVz z65w`MlT6S8mF@}wazxo(^V%YgWO)BT`IbL>jrfYT%Cly_tZPkd3zI3>SkC&sTXcFY!?e;qr&$0{!%Lh;Lu zF0yNJEi@`1_3}5bjRD%C^ZpP&$0g4i&j@0sFd7<Z88uppVI zAav$nE@yhhETSbSRoa2p6ZvZvsooL^W;&@%Wyes4#l=lUFR(eW&>Je>mviH) za~O4e$?M;qer&X={yw$h9JQ|e=xL=GZwvf#q_0G=lbkJ42`lFKFf8ozU5xR_Ih$yd zwY*B6FcJ*IPZ0C77;JWz-;@pxw%k6A)#)eoAKQj~ zN_)G5)=sr0=BE{BcX=$GmN53}pSX6^*I?X1j{IHYvy)V9n)lEiUj=7Mkz8J(4sWu| zI=^G}{1zBBIdj<~ge5FlMhUScjO3W?d?gVE6gDn;nnicpmWR#DG) zrHBTaX_g{s+pWuAqyS;lbWvoqc>j@f*#O^Wm$^EbD+xzGSzM%ytF1Kk$p&0caN$vY z(qU;02q@}ubj}o#5Ct;EbwCcCEqj#>MrWitWL2prKd@ja+x5IQ!V{-sYa!TosxmG# z(HVy=S6U-^j8to-V+y?Y43Aq?Tk%P+w#LmV8llz{bq5PNEehp$`D~~%3)AStqXt8V ztmT}8tfn<@7Osiq6~58`GN6%4Ti+mMeE5LF%igUkAD*SHdzjdFVAh{T&pLgQc6G6X z%Q}079_eU?=f{`F&$3m>S1CK2X~Rp$CIaU?Dzn!wGCY@@ZYbw`%Uig^?lgy6#CTs| zS4I5LZ@ zN)@bG>GRoR92JmmuPB*|o%~3P@`L7@qaDjl^A_5vURm2_GkYd)SQkCKMOizo09%Zx z-ie#T0lm;SThm)ET^{cp;RUnlbEUfM4~mV?@{jclJc4D459vD}kX-ffiXW##nG-oH zO{Hl^eFHz)Q3!b?EB=d)8C_os5pMyu-Z(=#CCMG-oN^msaTvh&{+L2;DOly?`uuCb!JO_ z&F&HnYO1!Jy`Z6i6H=}&TQ`mLkOGkw8Q5>jP5E3+9gYvfr&eQ#?~!o1fg!$RMs*ba zhNTc&@lI4$T!kXP5;HL)$Ng3{kWlQQ&Abzrdv15j-wZC~Oi`~Z+Ax;m8^Z+V(}^H% zfG(w{EtjVB;&C71DlDG7cwz0bkNl9w{G^38>Hyn2+^k<)dp(u)LY#%hsKjaZZVMw~ zPluA(UBUOEY`O_cH4@JI*t||&m5XF&yES4C11Rwmk4maHqVDy()yMa$%IuWW+jm45 zaX^1gg4GaXaXbkccGg8d$uqs2IJY2}k{c(^&Nkov3b=y77nZWUD~_v_H3f7mOeeGu zxn{C#x-+kY>2&R=GtEOyaMZ<^Eib3RA?=G{`(cPypOVZFot&vl*KcrF9#M&-58gbY z=fSer;4cf=3LIcT%pbrHHf7Bww4|GLLtR9sY-~D5wc}_?x#KBEy`4|?d8e1hZ@1v` zeuqE`rG3Na0QS$*(*d)W2o}f_T@l;kIeFchYZH7OF{qgm%?M6YbU>O*nPtv@X8_08 zfDotpt~Xbl_^V&%Iqva?gZLYUve$`!`ZDrI6eB?R^9oo)`uVnDVhpmu^vul37%_42 z3_jns7UkE6e)Z-4@$Z1cvu$|z7msuUzv~OzHGq7F-Pfe$4@q|5P7N=h@{Ha=GZnq= zpZTYJklM!M>q!5g_aE=;pDR!T`!I=AbCUa!+TZ>&aB%){0_pc)aySWGqp(Ggd=GKoAm1tXE&gWO*1 zB_g&u%rmLs5})tS`l-;SofhQ6w*pARiNDup(@6%)MnnNKE255>{mZ&YCR}iwfTe!8 zpyWF`kQiN;!OTugg_Bzat}q9W1HMEa3=+$Po%c|pwO@s>k3rzTlK27>%Nq+kY=(0I zxsA>coi{;m`Arz_WijHvtA#fBFasyf>yV<*BvU=ZTNMm~R~%Gl=;vhIE!{|l7Kqtg zeM+;ZZ6p#HLwu=$L&!H4MZ+t8jsD1aZOX!;WS;JQEr9e*8(u)e$wfLiR>v_PvnKMYHsjDcjVuQ&kx%x^rhryRvh6|COh{y1h0S*v zna1L0jA(1gLBqneD~laZ_SauM;_=e3y)B>ueNTm?&OBhq3Od_KanmQv2k|pFiCVut zMFza4gQ^oatwamRpn%X4M4lOr>nK`8pF4^lmBI}qy6oH6gYZ*f&;rVmhID?xnNvWS zvOJ2cr74L&iKrO>q4>asB?%?kW|@RgutsklL*W@ZXE+Hby?3qt@k!W!)zTe7;S|gJjmgnce6~mMw~NJ0dXJ;3>!sSo0X#UVZg?8?1(6~mf&PAmMgcF80n$+ z$`K-{@mC8Hl5fCoNJVVA23aj%LqQh;PX?m5T))@Ol`?Zs=it8G*A7~~Wv#wWxBKk9 zAaK-C7B*yYQ#WI$@u{sRJ-36&85+NH#9J}Zar*L&z08m}Fro+cgOiv*2s<-dAh{>- z7~qJq7ln0Z@IwYbFlBv;!`DpTyndx^VRu63jh7Ug|m6hR#Ac1cmCS>Ro0a&d(Um{dZm zTBJkcqh7uRX4;<45D#K~;nkdfv#$-O( Date: Fri, 29 Nov 2019 15:18:36 +0800 Subject: [PATCH 03/20] tools: adjust image size --- media/br-arch.png | Bin 31354 -> 28253 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/media/br-arch.png b/media/br-arch.png index 549b112f62b0652628802712647fc17ee955ab85..4569244ecaf61259d05b0cab9129a70968e11ded 100644 GIT binary patch literal 28253 zcmce8cQo8z*Y7A%61_(ki7rYe(TO0Eh!zpO69gH8(OZZbr0BhbL6i_i)FFC_-Up*Z zZ($6^817eoZ+V`()_d1o>#lYGV8(ZjefHUBm(Sk&oQcxWQm4GidKCl$Q9gd8`V0gj zrUZcqv&l$+Cs&9hvVj+J*GER~Aka1X%O3(zS_TX7kih+!`U6n;0Q&~;2eG}frZNar z8Gr4}>Iw+ta`0GH`MEd24xakM&B*{R@8)x&U=X?GBcmz-JzHgabrHcHmW+ZA{&_@K zsCYOWOxVLUW9rFhW8K?lX6|Z=4=^zbTNl10nfc%`e}yH%;1ln*;|ML2T`%b$Tl%%o zV8QO+#>S6B7NfsNy)f4Nn4mhG7e#RY?TOS}I%2bbV|%l7b91+(ytj9dKTr1SSIO_^ z2!HIa`kBsy{TK5}3MId6x6-=LzK;L8^wazF^6;3p4= zfR~(4|9n)PW+)vlnip*q1AK%Z8Mz$ze;WUPAoc(7(UHfm^!YIa=vP%5z?yQ>>mNL3 z$4w}?iAJ&WfxR!W6+sFgM}EPyFfa?B$p*S1_RBS}+HE9OJgqq0=i+jjuV`LV5ai$n z8xC;t+AXS1oqSi)0D0a57M@etd~^*V?i;?Ar9oPG5H58X-*3@F7z(@L4Js1MB>;h9 zF=zDJH$ma_tRPSenP28BSIl{y{sU-$U^XJ?LMjN|ZCBy5%!Y$KRs9;4tax1aaV!3u z@r6lQ^=Y`&mt>_2?BSb=+O6}GiFMbk;{fws=ZNz~ zY8XWRpnDwmq6+!Cch{t657`f$?Ze+{->JY3MVwCN2Zg+ z)&f7yAR=^y?-#__|=Y$fs4J7XKLTg=yzCdFZT71pXcBB=#$>kZCraOKz@GAzqv( zZb~V6=TN=nPc#yuoj&bvpc4MSw(2KXS)@F06#$ zDx7LZg{!_Tuo-#N5UA|4FJU{-BF8A2TE=`H)ssKgOb^2hY_}q5ZoK^f3p(JBWhWq` zl$LK};&>0JoaK=<%s1stQvugiTe;;)#E#2GxEG=ULDc+=n8>Gy=2Y%*dL>OBG}WyV zQe}9XFqG`-ZF}{@3bOc@7jF@{#^D*aRmHvF zRNxacK@%# ztp?4wIo`FH34+n*J_ z>)K${wGhnG$l!D{Ge0JvVRile&-y8`4&;?8T%_dINCZlkBoFeo$fWU)9OBOR&})+` z9QSwEBP(-8U**`!w9*P6i?g%MuRI%VXtj!2>b`I<$UM&@$_do_8DbS;KNvm29n zVaPYZ$c8RKU%|C8+4XfiyudcrE>*F>1tzfmwMYvK3HE0a&ckt@D{V|2;0a8>5|$sY z;qmA>F;Ab~YilylQv-=j<;WfXjKZM`=Cd+HP4>7c<<*i+esvzI+0yq8b*W=U!E~a4 zblRA%!^uEQRBH22qM+;nQo8r}x=6_P83msXnw<*$q!B zXS~ABsw+w+=9VnisRw*ja)v6>sq$<}YiWXd!$_VcSjAk5#q*6jCou&YJppyj#}fyz`f)4PlhFzoucD*GZtP-3?z=AsO#=$0rmrA*$k*|YKc`$9#Pgh%2FIO z#c?vB-)P+d(IJbwRY&&6clBRXT1i6iOyyc+NEL-J?bTGPu8rP zB;-9rympTLwS(qcyTp_bECbjTbFo;ut)?SQ$jtsp<+(>2W#V9+_xxnzeyl-6={-6I z9lp!R7jASd5MzK#{wcg$FWhvHx%X6~ z(%t+3h*V$J6c0Xf$Zb{HaBanQdGj<}gd^~^xZ19P)6r?0h^;D`#qe!x8ATXV=wQWm z;$EOv{kaPSi8&GU+cvPRrB+F$z6@x!R{$Qwsyt{_a4Ue4y&)0DA*F#z!o*V854gXx&xMP3Hqso&M zgc&^FE@z_YK*zN#6dxllURi#uT02{UTtMpuuZiXrgbsm*Xm8xH0(9B(#p&9KeaXOH zJO_r1u2Z2xi7=9epnXBIQiXHt+N6DMYtFA{8sV*Lz(X`*pRHknS?-X>n4Q7=V})Fn zii|iJEeS(k0%Kp}hq+n_*AkUd{?{>RY2*~bo`4ey2pZhIqF?8ZApR*96U_Xp|>@B{^<7K*_3%M zm=rb2QWF00{Th=x^K#|!ra!Hr^It4=3)-8F<-fSuw7J`WYgIPAhyF;vROF99w@5I` z?d87>chIGQk(N?~k>pcs8M-C7O>#t2G3AR92kZWe^Q6)IgGe*W8*tq@4ul}$h7eh1 zOV8hy#9dQ2n(rFGn@3Y!%{NESb}7R1Ekf3F{?wl(L_Utxr3)HW{ zpVI!g4Jp&{49;1X;+as;>U)=$H4LdS8vj`}mfSkN_F>$mypPj<%Du>x5AfsE^?W7| z@x&}@4lTL=>B1}L;!YoDg#W?B``dN}eD(R-)5>wS)P1(#NT!<{^&-6EwAV8>F!M&m z(7dmt(4lBHbPiHFNKX++LeQ+9OGQvbx9*LcYiRHk@Q9LqTOh?-@bEr$3hCWp0~e% zT@(ie=LD6kdIj1CnQ)P^m=C|UP;YV4F=*EQ!`jj=n$)H>7&iU)Vnv${I`W2XblAozvbQ(REud3B>O;0P-zn+V}km6o_?|V%04=mjH}bd zPDj94seL(Nl5x=KP6KDYVkt$2BJH5q<{e!w;VhN?ll=@*YVWWmk6tpWlC0YEl;D40 z-eo*2N}n3&D*m8*ED&E>E%UqExBp`NxY2&AJ3tX)G-h@~T3~{?OeA9b*3wZ$dT>e4eNem$R*sP4HuoUzMLd^ z>f`+2pKiZj4JeG}E3*!Q_pOUr$!Hqh_aza|WY>au{@Y5}%N!nG&sTo8{^L) z9fmy*Bhux^xpUziP78zf#YVm=9p;`FU)3dz4&?`t@&GWmw~dLC^{%sS5i4Bap1=N9 zdiysuyL=#!X0=(ulz}%_XXFz+&z z$#WKVI)|xKK8drzz-;8VH5LH!30|d@rMO18>Nb@JIZw3M_an8u{i|yC=h-u_5Nu8X z0lnQ+p6Dgk05OFOfk^&A6fwX%hBhWO6tUFWqH-Nt<&puACJ#9H>ad%Y#{di9;yC0U zy3*t;%3rV{B%nc;q2H4a9L)2VG4R|QfFP)d`+v@X|F09*uRv1$a{X_LJ(3~gzS)fgB zDzLUtOx8BGoIhv38el7!V44bBCh-fJ(@?BjHHlMIRLBDT&Cj8NJ79~alvS0L!$D2J zTo7&0V<1|C=&p$ca*9C%@9ww)j=z-nm~uu}w%1#4nTGiyEt+cv<)#}iTU*Ef5Ifuha*!Lyb@=z#LF|ua zlteUoKUSczR$EM#`OGH8 zzPDZ*y=BZOeM-sHP==h>?t|w1`DhI|ZM_#)Uu|z+C+3GJM~l@2W3GCoDO4s~)KAcl zV-1ZY2)^?1-Ibk`m!xEhnXJz4?&zqLB-wYwtWO+To~w>FJHt*xoK?`a@L@;l*#O@R z9)A+(!8OvWR8$S+&dm+Nugzmt@~=FA8p%1N5P&F60I6FqJr9%K%yMy-n?`h5r{p!n{mn|P}o#Xa`R0vr$0RBr$OI6h)$`bZ6P+q72d>ly( z`usb`R?IXeB7a5sEPKa0o6A&vTr1EgpJTFXZaRCcKKcdibR*Y57OoBJj%aYu{h--k zBwoct@c5jCnje%CcSkfP#E!3YqJ$9n(xSl0-t5y3wA^t z4>N%!kE%x7Qe=Hnr~yVLluXa#km6TOnq~dWJ`C*pQ@=8l6VGj0_ol#P7LGax9o)~6 z`rX`C{xwm1k~B2JUo~>n?J=_H8nD!HYKn}sZ=%K#Su*a0Y_NpFvSV0!aOdzyaIsKk z+MmT%TovVT2X40kQkn$%PNRPPVB+c?Tcent0(Id|;Ps3(QqbMxaieHGdXKk1^e*t- zp+tY>Wg>SzhM8?~D5DYd>)1)wXcH^H1Qf@fKx2AvGsv2O=Uc;P84ry;=*YnU{3!1Ev;hsQNoUgWIqBH0xg?i# zK*Zq&S~zITx+f#-6Vqo7m03-A>Hr`(z0bEsAn_ZWot<5yZ-v4w`U}G?h=bFNjf@;S zz{o76k5dKS7EegSf=Osu#Tl{R)OOBLuZ?v&{={H)RCj>Vj21LFpF@wA|G9ANTTdrK zR&|G#YEuq)Q3-FW8eN-MCzvac7F3kFP39Y9|D&8kO55b9*#rG`hZr0Ou-XmH(e4Bz zyG8u(AVLdq#?$#LE)>H(;8W$Qh<~nSTIIE!#u^MPwckuR(D(E zF@K>E$DS=AD}eFZJBE*(+vs5YrUQdm3-5wOW|VLUY4B+`^lgluP>m!Dk6+L!hsm0! ze2y}prYdsh;9^qiHZj8c3U+dBwQB~k8R8F$5fMoGA%O=(e6Q{73Y5mI;k1F$cOw^> z3339XdDmo>XMvsTQGY*DLdU8K%*mKZp6EN{LVQW4={E;U*UFpuX)YDjAnF1=>5M!O zlFafLdsC~uQL)~GgRnB)=ev*Yn^6eg@pFW) ztbh+yH1XHj4cTwL>I(VjT(T<_fHuu`eER}g_rb#A3FS?&<&9{bm?vv>A5ZuCd^uU$ z|HM#lW@5WL9v+=>qE+qI0(v^3y6k;o$>?k1%}u@iQ>}4&dlRF-s}CH-ICA~%r8Y&g zwYE(TEfTW>i|@4OOK1fYrkm|PR=3EmoXae)^A5_)YRO&Uz;pCPh-}^}mpofCt%G?) zec%>qZmLe4RrlRy_|!oIyJGgFExsGHESQ`+e4`{C9yibx*Z1Xg)3KlowN>EJUgh!z zL9d!A#bt_x4mA!>a)(8eMHp zk9Z|ufpsX^=GMo4Fg(cxFao-8NbJfd%sAhEEVlZu(TM zgN)DaY+Vc%$CCVdsi@RocH*%K>HTQS!Qzmo2|!X5Q>}PWK`@eaSjTy4_@r?X^J}* zV*Gs-U5-tjVN6&=Ah)Ud4bI@k1<2^FCZbY^Mb(b$zR9kP%b$fRUljfD2=`agtCTOr zn!AQKbATizVe3v)_3@sAtDn-gm%sbQ#S6!CWjBikk02?1r^Hp67siF@P$d*}#sXjZ zq|U!Chw3wHRtyM$Jik_K_lXUd;jJiPeTpmjDp<-vb4j2->E4pykqGq-opcfVB6VTTG7IKBSBN%cJ4n_Dp*cg1c|_5p0BIN44=-35r3#pNyKs0Awr;`0iEj1q-Z?vH zJ@Wzdr7zmM6glD@b!ugSR5p8r6g-JQBWD=n*(${5YinU;gJ8wMDW3JjiSZIc3laM4 zjh?e3q)?U$zf046#OJsyZq;jLOJ1X*bLI*3fDD^`vGNiS{swLsst4CL1^XIRBd6sKK_ z3eH=hjRDhWwe{Gs!8GlQchH{ZG6GRsST@N%sbpH&Yz;%a3Q$w1koxwtRLtB-Dz+|2 z2%OS*!;Y(P>Q5Q0$8lxUT-F-vb1LEa>L(etE{NKGZ}f4S*g*eG%SG*KEu-A}+(>|% z2|t39?YsF#^9iLzcXI5??c@nm?^;v72hO+osq;-O*zm9Y-VkMe%>lWQ0UcWf7f5>- zRs^r*ZohQaa_j2O3HyYe zXC(7GM}1kOCX#I;U89zn5#=|sOW6+y3tM)&ZQ|(1A2<@CEVlDnjkY#G9C3bBUE9_O zZ*u^Ql=3JPi#F0;7BmPefl5Xnp4)Z=bkJ&9YV4O^!$(e{F-^QX zw}~du8~fGTSjWUJ^wwN^E(2+7mG1&aRDRVx*Qm_(sUFb>@Ppxf=*;O{#~PUBlP9{b zvO2(6r;-DaQ4-{J{AEABxpw3TTKg(h>zt*m8eRS6fqrH_8UHfIgLHSRU}sXVMpR4~ zI?JWiP`lUUkMCI4d{&nQKAEX^^izHBHkYa0M_PF`)881wS~8FLSPf1o$3!D1M&2IE zXL#Kldt$;tZuay&L*nN<_52bQM9*qKszKmj>;0X{T!X-t|LAdr8AH!#L!qN#O`@-_+Y~O>5-V@)xom#BlzUj z#>(gm*0QMyYA3;_q)zF<3sCf_ot(=Rh5d8I3@DstDjw?Cw8In6yz-lEWMaeW9c)CA zoHB0SS5Y($3e}_Z;~cA zJSFZ)-0>U8#_6w(^}{EV%UEi^`3N7E2*!Bk?8jg;Qw=2BpVu=b4yCaDngd5&T0*Mw zMD6VI9}2xA(6rg*r*8rbBBBQ3-hXX)dcPX{iz=*r9J*9Q!AUGAYkzy7PfQrTcwCr? zi3Qs>)0Fu|qs)d=UALv$Lt)PlCwhF~lvqr4K%Dk-%Z376G)H1h^op!wgO9d;N3FD4u76x_ z#x~;}G`#}?I>d*k3(}>l9URk+orXq+(I1VtP7sN3ctrjwQ*`}6QBJ8z{k(<=asnDy zr;wdBxEG_dXw8ahj(-(zRWbQJd(CUi*KteXy|4Y4Q}nUaxrK%a&Y4RK_kHqwc68%I z7|D0Nt*i(YiT1d2icV&>Q9!x<#kn3^v2t`(2B}?vWmI)Rz9d4u*6%*8@CX>w#R=^r zfn94NU`E9_LjJ_X%UA%<5fv_<>K$>%BTtYjTvTwH>xE~Ha^%uQFJ=_g;r>Gl^UvoN zrEZe1qx~XIL5PCH9U0JV5&ke)71%aSeJrf94kM zX)0ZmtgSvS6%9H-JIZ(Xllmyw?c*Q?J)9ufZg+v++#-t(oQcOACw;wFup?QS*g5(O)-~!ZOs?alramxN z?cUXJf>+Y5>{o_+mieDc`^}G;0a*&3#Uu*x?SF^!39;7%!;t4+;j;@R{T2J<1kvTOJI#Uix zPA_FfB;V*3R$Y|)p?sS`vu6M{hZyyKf1`!1W|cz#=epbo?{nPK$(fo$-RX3v7^W-9gq!FkGBPHam)twK-r?>ijWB2IZFh?NvusOP z(hm8uZgDeEI~1|JbZntNPqFo*%=9HY+Z)uvKx-w~*I_I7#full664IHwxVKUUg)p2 zd@om8t7{aT3o_--wj3KiH;yv)b-1VruRTvYZgM<6IyZF8deawdB{$&aS9h+1nJTMR z*ld-a7@einU>GzzI%(2 zt|30~fvvMg=~p$x#`D(}r|!?DWz(OyTBEFiYKDS*DmCIH#~l>kzJRSLt2S%t9B_tt zSol}O5S~1k*_8%E1>y^KZl1uan-)#AP1s^TVWfCiK91EtXmS*a-+tDyC7tx}c)6XTrmF8{;Pryf zgJ{=|1Lh}NERCi!Ys)(|%}rTryZrMT)dN`y4cC)BRQa&p>iF4+%ewo!v%1;zlq4X& zHU$Z}NYTpOvz=~5X*tu(6d>go`#cXklam`hKn;@QcqiPd4%y@5D|fBU)-=%YQD|yq zk~V2F65=p*y((*>#ows_S27`SV$|MwF&{noVdO~WS7;xF6QipL0^;87&H`^vsZbDy zNT7`0zH}%UUa>pVQcq;t0XM!E=q*2Lk?bN(GGcQ>NpG=j0@Z!(PNSrQ9ur~fA+(hF8K94W%S2ZN&=5FpX1M=&uX+- z&(GtGQ44D|NhMpxej_86&}cqWW{ujKXOzC2=0-*`S@CH@n(Qfy%<-yX4I%}h8wF*D za`zOwMhBAQ=4b(V_3e-N-i}YHcoG(+!JFi`wzXZbM`P7 z;i5nZX&riOf;wJyJa#HNOK^eJ#5fxJ zDdwujm5ud{?vyGerNB;p9RVrKc1H*%aS!Enaa&e7BWt@U;KtX{>ZIi4WrbZnzOv*_ zoC}rQE0m@Q;(MOfoga9PvByXxt_j}210SxWgcK|y=lr_-x80064@U-MXUwqY+tDiObaR&M+j?b>R zE`66eTRMxO)REdjfrSg^Jj6fyYDJh0D$kGRT}S`8t>wUJd2xh8&U}3;eF&+MCd~3$ z@c%HBKx<9bSE}glrC2o)8s$@i+Sv;X^lJk9b{*$niam(8GE@T=xVKV%$!1NtkXr~H<8pJ2qCEOh7uuX)}yB?Qs=gk$ykQa@i-Q#S;lFP9^|t(OjXMDpi-oXt;-Kz!;B;%3ZG zlGiix)_;6Wh7YuDQ1&CT;1RS@*uk~r108{~5W8MY=&~Eun>?S$u-Oen_XMd+1qHSt&6fvvEI`jI!{nS$%t) zkCY1~ik9Q!bB=;|WVF?LW5)*i^1RPEeFd1LxzA$sZ0&g#VF}Q1`uqtm`B*_DH$xc{ zcrX1!=?!;pv@}jCIg{f9jp#5FI}s9~tAkQht!-jVJkv&Gpq52Dbp#}~Sc?WGyay|G__fGz#gO{} zQD>5iM@&|PXX!nwe|ANy?5Q4mTgKPZHveQQFu}F^4lRax^tp{T>FcUR=_%5eMNwPI z3}cyH!EAON=tBPAM%y(Nhn62R`|A7;Mr}^iKq;lp{BYD$bNq_KWgmfUbbXSW;Bt)N z6BwVryu{s=MP_=CtHAV8^?Hv*&{K78$gknP+g(*#Z;1?P0ItU4l){ZO%B3;95EmLY zlQkm3mc{BNZ3@y9X*wf?LarfO(u^5$L$U`%M+y8LW!oct<;Zm{BZ;%T2@;U2>#&ke zu@WC9nfDoM!2>|1++if!#R&{v^m_IHchZfCiHXps7<<>>_1Nmn;CI|$OYP~-qaC1w z`4V_aOcHLYY&b5COKf3{9 z*J5fsPVee+jfvt-XFPGhtj?U!$PmM|I%^<+L{(q_H1(z z^y<6tXxpy;;jlV#TKDW4Z<57e62iXyAk!H;+O~^$v(l@ebP}rcLvj1uZN5P}U{jAd zhU$dG19qmlP_8w-@H552h?O6@D~S;S+{s=@2Gim7k8&HEP{c}htzR}KDd zlZfR@j^f+6We09CTsrJ^ro&$xuJVXG)IUQQ?hubnyvd-K-)yu@vj=)b9HZ;^1$_6k z`>(W#-TMqoX>pplYp0BZ(ceivwxRLtNgBw=Z9a5txjh{YSf6O;q@AK_Fny+M2gV<; z)0{8s#i(v_G;g^1`;CKxs`_hVK+W7NnzEcKY*qUxwX|ytu!3(U!{pY`1r=>PIzNb} zgr%3Qh%1*sPSsw)CpXZ@-Ve<`FxcOgVPoVfj<)&&vNZo4AjvUU;YIJB>p`UmKsi8Z13L!6d_3;?l6Z6M!%#2>fc{T`ZqpT9e}vH& zV|Ile#q22%*C@}8mLzqqfNG9pRU05LI93Ubjit=w9{9NaS+Js(7$yKt0JahEKsppP zhO9RtWNc5=f-|c)IcO$TzC)+4~$F_6)*JJEKh%9_W4a*SU7nz6c8uV>Xtb4k3d11V>C9GWMT;sHzV< z)JjpBJJG8qA=vELAUD-#EnHuYs=IW5Q&m+}&ph4=h^MIdT$}j8-tdo4Gp1!7N~YEu z8@SlvTQz5YECD#SLu&78bn$TZ>T?N6gQM$nAA|pnx^=jd(kZSse5jy%?b4+MI`x>? zl+d?h^#9DDNO zv9CDL!*lpS&#yI0pRBqpA||#)T*()m%lvtQBsTwjc`X^`#OE>C2GGr3&k6UHI9vK@ zqTTBdocpBhq`0deq1}}|B96LGDWVZzXk@f;Z2y6AftvqS&!3nWjq8-+#&F8*TCi*M z%2}V{*SvBcC&E{4fB<*oi^h~G?_Nf?G=~KSjoj9=t)PcSW-C8J*>$)cgogJGhA1U@ zagJ$>B>}BbhqS~*M*Q`$M;XsjgI>_GuxsBJC8K05_n>M-gPaBcy#@d3?0hJfUD7qC zx^KkDVMvJ9b{KLs3Yk-am3qWozx(oXrZ3OqH~3!VvMA?DD=?)5JqRUm^-$;GB(iD@R#z3fjYMyB(O=7# z9tV1PG0K2&o)7mfpreeVh+as{O-=paC2)rrZH_wZoY)j~bkK3!U=4J06UZkhv9q&5 z`1n;4JqfVIB=8>;>j~zPWMmEL_Aj!uH^#OQ_QKup99pDGoBX)`Q_sQe zJN|xzt64)ixm6y`nHD>IOj?R())`_`l7KqK@a^?uy#wc3$zz>$6i1aPFEBPIj2BunS0sABXDL1Fk9n%oG2}DRrckMtc<@UJcgM)+5 z)AzQ|76R1xBYnCMV>(CIp+FfR2@?N7pkXg}dy|ODSA@aN0|@G@0?3oB{tDT$-YSjXX9J032HqQtpC-5|yomFs2@TGmGa ztc3{!N*RU@09g7y7$B;v-s_O_F|T75&>P+J%-cWr&eq*c_Z%gRsEyXqEp_f|^Cr!r zil}Nv3JzgohTpKf(hiKsb8mIR)t@EpFrz&2+n9T^nj(!r&nRxhb`mNA{{?>Qgkue%EHxlC_?>`aVEcFUQY^r{}Xw2&7#AYY(pPfpX{9D6=OvOTlf_ zVtVp!(~Ht9J%Vn#zC^?IIEBQytk;y@zO5cF;U;{mF}>!gsPtj{mwf{UNJsIbbk@zl z(IPK8aj7s8NB}e%F0@}BdoDfMNMN|!<1`pY5JkvGL@F9{LaE3X%B=PsxU^i6n|7ivezU)!SrDJGB)vE@#;AiOV#NFsomVuj=yQO@r zynD8M9`v#F`F6G;t$xd&T)uI6+$1r)8LxEi4c66$`1F~)eyr!Z&5sx}o^IUrf04$d zaO~c$co^H8zWm*LA$|S=rTBsG)yAT*hFIg?!C)giIVHN3a|ELQVO&F20_*|(JV4Dn zRo&-MsNpcgZ4`2Lv|e{l!wq9Ube~8;r{Wf`An4jnWOL0(g7X!UNySu!=x;d^_q1PT z`^e$8Ax_U+UB`e@6OD)BwV(n1xS`>9Hlrf>sm=O5{_gT8HD3+5P799_{F)KAfmw;+ z7S8;+%A$`}t&XaE`-uocAIie@BQ(9fkkG{4)xFauaRkwq9JGDBI?5hlQ=odWgE~A9 z`j#rq-6Id~TkTO*ak=;MOfzn5| z{zv4&Y7;p-^{%CLY4mY!;(DA4%8R3>ni{L`u27B@xl(`(4L2*_+jgrJs!3`SJz6CI zT^H7a(o zdu=0G5py<YM$OUz1Ge+1RDqt(*5Ke=AbRY7=oc2lEM zD(t(jdSrcqz0LeuGb_UU4Ct2Nq2)ghKOH+!%m67NPw`a1Vh$s`20aAkq%j<1F;o|;d}$- zV|Tji>_-#Q5|8h*_p0eWyU?}Fr#tE=57S*B%u8ucJj7T5@y^L>+w z!3T3?2Qohwo8Z?L?*tvZLENTCh6@~DLz$lj8QRy2YY(*-@+Sm3A6I_2Uc4S?k zNMA@}JxuoV<&*1_{ZA7v_OO!{TNc2|pIe$bbIoHECe3yRSCfN|-w~s=?vv!pC|LCZ zG9grEx$mw9G9@PD@;Q!GAzIF(pA|qOwrYg&Rr8JxWAB0~fXlt?1n+2KRB1?tEsd}HwxVU96nZHPHd~A>ENmg5TjFOY4P_tWi!FNYuco+@ZNk9l$DZ%pHD7< zh~Y@Na5Li=6sFKgXe}z|hd1Z27v)%NFa^{^lRZ@+ta%Ih0w-svOV)(XR?w5NV0h4> zmGU6cN1xbg*g~1uYKL0UY`W5lHmt+z^d&IQ^Y&uPwoOJmw^xSa8&+H=H5Gex&Iu?# zg@*~)WqE4pq8r@EzM|mLA3-T(-O(-FgjMQ_wGHoViERyVj&vU?7s5zxUFDhLQf3iX z-+kQjlm5eal4O17-=~+G_78622d#br(xMyS52r0}>B2;b?SK>CcfJc@2vRJ10gv{w zXZ`ZF_`f>ofAin>cNES~mYJ76fl}mp*L_ViL7=;}b#>EPp#EC`P8QV&0IeoVvVww* zj1x~m{bHBozkkvG*K6kge#Ac9lo<4CuY=j5@fLx2a>6j?@;2P7z4C_L&0GKQ`Y++6 z?8!gmUqbE_^l1okXn=<9%RV3mu~-R;nmq-Nt#;|Ds7fb+gr437>zn{b|Gn%SH#j}h zR-*>bdPjPjih06pG$%4#--o8Ya>)7PDA^+qidh80eSK|ZwZF-WB>*FOn;=!gB{npj&*v?z%^c_{A$E&%52bo=7Hw|_6Ih2k* zYtO-}d{#nAq3)8EL-5mA(~T+P(|eB3q&r*xvIN%9yT-L>c;~Jtd#KMbV1+K7hwV}H z8syS6i@?PaAl?;+lPE8?<l_U^~f!+Df3;X{rNj)wF z0?-24Asq-^l3XMTL|Gj_eyxlj{~=eVTxBr=@e1F|dkePun?&M>gkcb<|5F?d5RU)1 zZvyyl!of3sKz(ZBU-uj1rI=j~=us@7T|j#MpP^n&gXq6P^_^2-6aR;A8Ms_R&e6Sl z_XckMv%kQV%=CA&IlO z>hLG-_C>#wUon9{BE8TE1)+NVAFB~pfc&()qs+Hf9DK)ry|RCGL6A z-(AD~T4bNzApg`?__6ahuWDkoy}n#o#myPUo&>r}t4sPm5Ug|N zmwQe*tbIHC{h*?#G-{9U)xa^E08&2Hq3B#WU=91TF6@T=r(iO zo|nztB^>4b>>c@-{^RtIaW}yqOz)-IW~uR|l7cKNIdmQFhd#b-WZR4q=jQ~Zi*|bq z>gV{IWP*a#z9m_JQk0wkIAzlJ-*_jlpw;*jU{7CbzFgv;hZ$Ly^8MFKt^Z}Rj(<@U zaNwGQJh<-5`*Nwjir_WxV=wiOfx-XV*d6~zJ-E~gORj-Tx#M}~D(FgYq8D(@#5~P? z+Fd?X^Y+AI9U)r%F97=giMn92eT1Q-z*i__o}ref3zAn2kNmY?4j$)jzY@Dfxm%>- zx*$h|R{IMqFH4nzh5sP`Yr752F1PUO7Im*J`r6~3ZR>PsX^LFH*DB^JRT$;u0dV(q zBWM6xsF(=d)9Dx()4qshJDm*L<{dnOX)7T<&IJbe;=dXk&E}pCZ@Zxq71q2tJzr2! zRBfa>lb$Vt*DiBUGjsD@I|GyH!v`F1iBS6;qhBO_Y46?xf3U2I^sfjh?1e5E7xMry zX7IRgvr5X|ZX;qI+k1{d1|A)h+hq;(uF=>X{)KW?bt2H}=*;5m)0(vr-fX|(pr$Fl zc07RL>JXv&D*&X!coqH(z0H5djt2Ufvu9Z9ooAV?_J7;}pLaJvwk_gUdXW*@_(ctTX35k; z02fjFRnW1M z!RTJV#>!!;Qk~?R`Ki>?lb`@SL}^@RK?z5pKtq0Me6UQxCPzub{Qs)$%j2PH|My2K z^$4vJ z%S-Dw%Y=WTiv^oJG3`#jw%V4q1xgK1d4r$I&K8hshOKBoml`Jz&D$q7Pz;96V`x~5 zLBlUspsSz)3k3M4ERuFy+9Vj*e0|aT{yoTc1L5cMCm`clZ(TW`B1ifOiBBjEppl&d zDfH>1ISOYcRJGhZ0=|>x`;Saz0O58hPRvPOUOvT;kKUkseb1gfb)}A-S!bPyJZCZt zL#PsAY)g<-EuGLktdW&LmTlN_{!hzt$;~o2Qh>mzAK9p>*y)iOPH>WAMWg zr?PQ}twhR7^xf_4CnLV>#Y`&UfQA;g(RBY>NAXf;z{{rc0|0)7ot*=#Xd^euIrb8v zc6`RLTe0|3zne{k7TJB9VDt3tjFm5MqD4>dduVHL!fco5Q7fHC0Sgjo0$`mw@6@IL zBJJ^$_0)pe$G*?ub2k+mj_N8VieT{IdM!?4k3K|n3UNvuR}ReV)G(9spUOL8q$CGw z+58Q+G+Mt?SMP8XvG;zd+O#*V@3`t#QP&0VXx9MZ+Gy3<1I}^iJb!WewK=*r2OZBO zaylI~%ree}Rd`lg-fuZ?ckpr?bwuBwP9L_QMH>bP7c z$^=u-fC~lvZd>|&d6{fi)05_C!pwg>wl~@PvcNDX(-Ly@N%eKOS~IY8_l5|;Qxss+ z^-tC-{qefL>PZUv`UZoY7RQXF6V6{Zqp$b!^J4@iou#hS&ZJKhT!LjDo8o=~QT^`s zWD0=#Ee!3XSW46LK|9u0>SIRKgAGEm7ZxSL#_w;`$sFHI@>j1dSN5BAKHhivtR5-E zZ{b@;WQD!|REjUclvj)WY@wwWs_BD<*%`DV=k>1^jKbPrKlzL<>X~3(6`(gKf1v&WvEji^^k7PvPt#Pg2s;y&J zT%+L~g|2+48yniB@n(D&JS}TY9srPWNIK!@e!{H3>6&!FMVbE}JObYaBFLm(|8yXZ z12B@J>N%166O35j-#EKD*smPCZ{ZhyL!#50fHB3tyCdb)al;rp;@8&?n*c;}dmSXF ziby2l7`=w;;00W*!}4zg*7FPR@IVp{2L0nwVc%~sefQ4|M5e%&0{~5lHeR->PH@U; zkId#G4~K9Lo8+;EElNHGVtJCrTY7r5T3CM}UFQY{QpMC1LlEzHet+YOR+;EE$OEXs zXQ-(c!_jtR$ifOE&!ns&UX8Bgksoc7rAGW6^cu@s-`T1DyW5xEW*yl2nYc(IMRTFH z@`arj_6X>ag0RWihY9XE;anLu`#{vy0>X1R4+E(ruVZ#WNus;w)bY-9dL&DS7k*rf z^4~UiWwR8Db@>{QjdXx)1WIXNkxr_pSpNhp|04Os0e3FQsSW>#NB`iTe-Wbo+od58 z@s@wnoYZ@Z>!DhA-}()KVP6a<`qt^?Y_IutqNSoNcyp zZn(2yBZhFPq%6kJHa>`ijS5=ce8D2NO-bN|pKzjsWYY9tJDTrg?7}5)a?{1giQ;*C4y1u}o~3-b3TFiwSc5Mm(G5y_eAiFFTrjU~6cv*vMle{2FK z)#eWsHabmL4v$6R??Pep2TK!IVX?NK6dJc!BZi6NiUl@RhwE1Rt>xwe7FtRNh>1(t zovY?VHo<#@(mIO^rA&tf~obcxqx zKWm=S3Sv{SF_`C+`|(W?%skFwC%q6=6%dF&zGpzt_#LY%?V^e%J7B<&r6Y%DFQHc6 zEV1G_Hk_576Dp28jkBfdtxVi);VXfyLYmJpi&zj-m`?1xqM%`6^n_SgwNQgAR$+U? zzCK!MFGYR#+MGp%vBqJe3!L&im2CVipEx{rrNf&ww0fOW(J;0*@NixYt09d&(|gm_ zt8ov`#eHQJnX}a=RcE!56E+rn!NtI7Z){kKG!;TCTj;^D^#?ARPaUegW-@>1+=#=M zh=8~BnZ*I+b$_}CGImV&??I{8mC)!o6}BDc<_d1^P5{)}3du-HUg5J#*FLBWPfG7N z{>{ryn^0KEDz`=!NQnndClZEzf-`dFW9nwA@#uNxvd|K5Sj}o4Dpq@S06PFvU3ckA zLf5GyLXCPPo}_((7T1z(mWR?8?waGPmXk*VX?xqecztqBFbhPTmDdeL!f}dsP~T%w zI9UHezTe2Lm*&s6Noy|ip573?vF)H4|Fa?)7YtE+uj65wJ7O7ZxcCv`aWdx&Ko$^z3;S z)rHMWPeVJMud;(;Sl%cbE3vLC&!%y3^N)JOLS~kV9UJ9GBYC%9vgQfBl4_X@+Ph~! zmxNt;*#&_=KM&g+%2sT8lcIv1#u}-xo6^2jrYj%WiCgNxvOaV%SC<9|2AuI8=8hr| z>D?XOq9gKn*?qd9O=}IYU3@6Pe1rozjJjIDLfmt#Z%FN_Hn*LC}xvDLIlTS zTDB*%$s_y$yQ2F3`F$Ya zXN`SZ<_|}=nDV{wdo?GCVlD6=3u?6FF@yA2EL6|iB_o3 zoHQ2ct~5R7y~JHfJ~aH4W^Z2hOS*VmNF3eX-j0~)N}nX9$2xG=sX3uN5*qAeC@gxF zT#39#zS%f)iI5Uew~)=mAMoh9`$GK@Cc&07qk*FbG5Jm4sESCJB5P4jShQ&BKq(}?K?BvsbC@)0)R<44ytDeOUijitz zGr00XeSCqN&xCkL184C8bBFRk?w6_WnTti|kHh?0oZ#Y6M-3{tG3=lhCO(yIfWjoi zX>o|C_+B+_Y{06ccCro9-NDA8zhvLTROD)FnsNN~g7Z=a#OIlq<@Zu(0td>G0kI05 zUi-a3``w?ps2uAc3D^E``l08{mSSG1_|tZ??91vojYs*zc4_>WLNPfn1Sg{7k?AO& z3ohQ=m8$VM2xQgYSo>P+xXIcMZ10r zNBwVrVY6g_tC9kMIv~2m)n-5x$@w2J_#%$$OZwmCSMwc^Q*2t#l1+y~^Y)`M1ox~r z|Dp3;8tG|@IXU$B$gHQgtR|N3+a@6wz0?{e7TLFw z{Pom@yvEUJIlA;sJz6KCykOuO^0=B&uCs#5^|Jx1unMbWISfI-d*Gok(7}AAGh}s1 zdvtzblYg*W{RB5-?tiUF>Ky+HOy(fQu>Y_;{Uvnnz6(4A>YhD&ru08GGui*Cnf=p{ z-}iDtLAR0v|3b9DarM<|-y%PR>L`CW!^P#F{soC(7WzQf^DB>AlV2Zk5y!vCulX{N z;H06o+>qatac&CWFFf}5u=~9!zwP^c|1lY9_n+Za66u!QuY0nR*8ZuFLE3iRR3>-Y zM`~5<#sOG=R3Lfk{R94d@IYSc)Rwf|q362?%}N}Qw3knQDTP2)%qem8M8|$Mc6Anu z!IYJG(;pG%KgG>wjYmp#*2O+_xmn1e9*84|7rR2O5l((oE{lwhkB2_BKx$d9rfzHX zEg>2S)o@y3<>;(2q1CaKB4##ww0$vvN`#^q6H=@Osj`y1_)>ZfHAEtWeBJx8-g7Gj3zs&l|O5Pkznh4GpY2J9}av zh_YUBgO!+~FIY2ArEqO=-h+V^Fh)ANVdG6RS`v9d8woNrqc_9fn{RJ>u1hNYtRpw4 z{^eoC5XH8Zy*vcNmX(#++b7+D8{MjeJs%B&MT)-hc;1|(Og~+r`Z*Bvr$FX$tVa?|bNQ-EU>wy4}JD(6me)S=A zfY@j$F**Kr&a+vSmf$*yBevR7r)WvO4pc;GvwE}*%5Gi4pFpjV9vai~jB>!RIMcXn zjYiTl0at_`Q6^|$QY#h$k&(`Gxz-`$;@ncw?5CQn`4W<)>r{X3v4eJUSxL%Dav?Ce zX>hggA8|>1qKr|xy{f7DPCQ?1f95ACFsp-k&hV0+fFO=jdamgq9un}e0-=|qR%c{_ z)d0_0=Du#$HXWzK;5DG+1MCg>iJbYGd5p&PdT|eb;8oSu4<4%qPusrNJmuUG2}uzA zeUEzpQ2OP;=w9=({HUH{2>tD7?&(#}=!1|?!W$sDxW#c`J~J`?>WU(NzvYYrxI8_0 zdMkLn)st!9!d$1!`lmprX>l^HzqVbhXcISP#NC23UN@?Oa}@(`E6E?&WC$skyL`bv zf@%%umf^2sl8PetH-1S4_Y8rn|Go#DK?7$0ZY7usK1S;j&Ys~FZs6V}cN=s1rYH!W z4;ke3rz~=~sf9`{O08oP~ zL4&w2jNsj~-s*L~9lh56O#tcDa=+WPtX$XWdmpz4hzv$i^YrU)M$!?%3llxcnWTbNF@+>d+lZ z8^NUdj#ytw>omC}6p6Ky|2*2Wzw1?sO-FXV1luygb8M_Jpbc+0G)4C{^`T~&Yg5n^#r>oJ)n|pVF!SzXH{9D8;mwq+x^i4c zhe&`k$_hAWBh}*Uo^Zn2334OmM@@$jnpiSjt$bsue(t@e6UBTSf{RW5HtJk?K~Qoi z7;LXPl7{`?#o8zk@wD&_JA_j@wPGuLA^Zy&Lv0H5xB`)QeFi1;36qdqSeoD)W2jAF zuu9ZC)a(|9&&Z+Xl6Q4dzD6~xTl(9AXuBt2&qg=Yt0mQZRz-P_f9|zh3%vgz5|wA$ zJguC{M`b{-4ErL-{3M|HSWpXA<83F0&A(?WlsR7bCK>|yrr||0$PU!gk)0_q>6Cc8 zL)fPRVdV-+3F9}&Y05a*$2=vLuT*3}USCMWO{i&J zS*V4^rdqe5?v7QJZ;Vs3tS-FwG=Q(qlx8(*Tut#Fln~AIe%JL%{QX;jVWj)uP;t)+ z{$iy_A^lCrl=@MJoIiN1oHZPrh<=)@B#x4)C?dLKnbO(#m=6wYTs4ue;weD9o zdAJWUEJ4k`W6#AM%8j6-;rnliik-8$V{0X3cPQUJU)e~`tg=X6-9`KglC?+-92}Xb zoavFYAn@**4d)nL)m$Eo!sFCF%!5$mO6Q?K?SEsN)s zTnQTx?_wT`K+eDJDR@`tw5t5>z_mb2dUTP`;!R1(yQpnjW6lOoI^R3~&b$~ik;5#) zTaiWMQL4nQF>f(R(HJSOej)cOPDK^X#~bIQuGAU{#iY(Z*vG9g+~=oBkyy9$fnlxD zd7fo>QqgWV2OfybjSI9yb1_>H`I(sWBi|2M35i7dV9Msq2S+a7yOIYSXI9h?kYK3< z4lwDkC+KO+RuD~}hST!nQ|G6*McN^?Td<3*2P^2`y;<>(;bFW2y)03$ov17nL!|`k^n%qXGB9I= zDUG!bFMzCaPk|8%1#5H&;g`^@mM*3LDy{()o56AOI#=y8n} zBCT95Zuc@qxG+0+)lQS1Yc3u-=v-r{-GAeq(v6XnkSV`6H%1$xUdJw@;4dDApMfjZ z9T}2SwlLD!Ay#JUV+EFz4FEAVR1E5j{&jpRigsT8h? zS?<#dzzT)!2Wz%!l)f{@TB!0e?K$hdorn;gj!h0!YbZu)j2i3G?EUIDMzs(25Psw51Y7Bk~%rSqtXR8V^c#%C( zZb?O;xZ$kAItbaQTW#$0jEPA$1)X0#BH=vy%6!1onUfoL_u*R~*_D!#93SUG<^aWYl^QB4r=%EYf2QIs)S80|- z@eK!-o9wlfCyvPu^t{qtr{S!bUzP%jg9aW<&~~OxKG;Sri0!kHPR?Hssejx1n)cmF zY*hVSV3o2%3Xy5g&VSPEgq{Ib-EIVzRM0ngrHeIX!>W9zpfQs6g}8?rXrID^VlG&D z0mJP480Vm;etDSt`DIGSUf#cMx4i!~(|eHdqfxTvwpLT%HjgXlA{XFs@HkE+w0$K! zYHwUk6exrCE4rc8nB*bxmBm45OVHW!vd=_t2uFD{`2(ZLh1mFt%Wbk$PM+;mxjVA8 z^i0YJdZ1Wp@H&Wc4?qA2@kdnv0c2U)-A4PDOGc>AH0un*EF3AB`RrK4p!#+xA4)lE zJH377ksq`s@ZJYD(>djuTWk^Dd0&2+*P)O00h)d7yshsADP>H3kxBvm_7(*+ic?1nBKw3yY%`XSyLM0%=wa&3Gd5s-z6==hm3@VZRbvT#46?28Q_y?rxp3bli&MH@D zcD%gIx3$kRA3YPnZ|!_9{LftEo)$2VY;BP*uNO`^G_k_EFAjBjzVt_!z_3xvNV_3t z$MEq6wy0P^ZKKU<$WFVGKFL;*4Q-~*HHVEo=h})S+8TE%Hce{95SERdgHl3n_Fbgg z=xQ_@$#ce9TF&y@e!^Qy=Fu9C+3Cxgscfh=u4adnQR~EZiN(ZEXO2$}3gRyIHe8f5 z6Uk{>;p4BTmDO}v%4uQErQm^zktaHj9v3r*u_QezN=?S!dp)6pEemU1LN6@t1dr{xd347yO z-w~pxakxJDN2PwNV5)Fe?mc-U^o-O%4*v4n$=Ap5L5Br|fI*LAu;PrG+2KALpXzgd z39vxq^_so^SZl)Q8x@X6lepR$c7u)Pw6ZviYD5 zOsaQ+5mLrH5ySh(H@Oba!*9WQ88Z1cFC~JcnEKB6AtX1Q3`3ZVX=!Ma2 zS{b!8f#j9T4UHN5rWj_MVKN)@tY^osYYgP~cvZzzSP#Tmi&W}U3D(thPLEmj_39gA zt;Jzc2uy*YbI=s-hkSBaU8?xzNg&bpJKT1h6*YBb*YPsk_r6a}lxHLJ z@G@=_$c}u;(-ua2LLS0gV_JATBKPX-IQ0O@89gHJ8lEzeJ5?Nqs62{1f_j?EPK=6P{J|LJ*jXyiK&izcskFJWP=J6@t* z;;)unmORmk1lF)niu;%K!LR$GxpErWPQzJi3mJDi-L2YL4iw^n*w*{8!*QUfA?ZT+ znA^I3kZ#X@E1S*5-yZ7$2N(q^7ubKs4ZH>q%pUv;_iOmfeECVXEY{a?V_aaX@9w%T z4tRw{HqKdyMY!}gr2xLWA{*WAdI%-(pW(gD<>3%TiQo4CPpPA=(FiWV;Feg(u5q_; zIDpb?n}@QuC?I`G2;ehl$l$CMZ~V;+K`O#@NU}O$RhtBIw15}^srUYUkINuHnjHv^ zwymYyb6aR>#3dfT$O!im5nY!7+P}rMD-dbna9v@2|Dj!6DS3^i{Y&eMfk0&aH)Hex z31Yt;%{{xveNY+zQ8WeC$0J8MD8O}S-f)GZvf!)WKeikEb=qHb1LDs&K;ZiVnOLS# z@?$H;gdHk_y)D7(ud0F_(w_q*xGXBz8TWtt+eAQkgTRltRo_x$cU2Bs_z|RnCecex zC3~>Q8YcE8 zpjl)AZ_8njDh~<&YLBX|8C~dvYQ?UiP?Wf(>e)iQ`M{C(SPc%A(L05ZXV%YWFJZ7t zQw6r^hNinzA>zOf*9wF=AmN7u9av0QP=%D}Ijx(p3CJfx9q|Oax|@zA#ZGfLX=#@r z(K5=fgaLP~Pp@Vqg$g+dziv!+t5p~Y%s<^7JZ)TUY<+$$#H3?v%w?FRTlrw4-Z;4x zJcpfzl&4%hDuwwmtiKs92C8!Vz!9fxYN^oS+DVSeNT7}X)NBOJDE=w0SofK`u=FN} zqkDXk*b4lL3PAo?Q{o&f_&Pmvzvgr#Wr_6doATsBjPJvn;p5XH|a_iLNKwGh=vAL!QX8kMa`fJbMBk)!5RaX! zG^jqb!PLe(hD=$k4#T+=Hv@avU##D>hs>#=RO6!r=wReAo5h4 zA;qVR{HlWOf{nmoseaCrw^K-{S~LEOaCOje+yQV(qN(8t9yNi|=Uedo_-#yaUD!K_ zK{yynkCZV-U;OCrjx^c2|2oX+xRkuW{zMMe<_8R{3J?wXgP;CmvJ;??n4Hcpkt1`%5nW2VbFk!Me^r7LH;+)88EY_}8zXEuq$hzX^>za&nmot_smRZ+NcY I4E)dk2WH#_HUIzs literal 31354 zcmc$_XH=6<^adC#Ac~+Oh9*(zM5>5%lp-PsBE2^eklsNc0a3c3l+e5MCP;5VX%dj$ zdkc{oNN6F1?91=J`~R}%?AZ@{&VGQvn|WvM%-r(aXYPc*R#(1$mH8?N1iG&BLh%g< zbm=+>bfM@9CGce8-k2cpLGALwz#Rm-Mt}Z80ZK_@1RhejzfqP4mG`r*1OHsIdG_iV z2viw!?bPBj2=swYMe&)o55+c#F5Y3n1aiEy(|Tgn6Ir!Uuc-ONQc>y$dpet`m|pYw!O>|zZ~kiDG1=f3jSxcK@|ETH znT$%a#mmp#b~AnHKfauNdd1vCWgI@YBZ1ub>6}^aSKQdxR7^}O9{M$TP(vD0tO~C zFpRT12W*xjdCQdxozkxlML{>xJ?^T9H>ilMrNCMhx`! zVsZt#I>ZD?P&g;Qv9C39%d>StRS=AyFeskwN}5P^7M$bBKpxv8$Kl(5JSXzl)B8V@ zdS}8X5+f$8=J6UbkH?i#IxUfwWJH9u6$D?ED+|e*`vmB$P{$^Gk?JHylNYwO(}bc^`S3tHt-HiJl1F0|6G5_efO=mFc;1gt+zD81nI;h{9I4 zr2#C$b5v)meT-YT(IE_S7~E@Phj5Fml9@0^GoqH{M@w)He%Au(|3zNk4L2YzNkFqtZ_ z-CO5xXZW+=1e~~yGAALWiuDHgLeA_h@ME`YJ;%Nk0dh*aUwhetgP!?p7ZsdX7xSp@ z8%fHy0f|Ei9goM^npu=jyhoKhiJH8K+qQc~l$e|< z9|}_Zd}iA=`eKa}4;thkjCafxo6>~iKctPA>ACM&3FegEKB-fsEwWXX;7Q~bAWT=3 zUB7P1VjC{e_rr*J%9OyZTl*!3VevG3y3nNFpT2SK`# zEB|#0tD&jV(-iC33>QIJ{w+fUBlzmH)Q7tSWD`59`rbIEH)z-1xcn|=FtGA}=>V?g-orO7F%Jrk<{S`S`5AlEL+KG`v zX{JoUpZ6G&lbkyj!Ae5LBK_V`*!S%9QK2fq>>1hW1=k)tZ)^J|MPQp|sA=v|c z1ap>C4C9sm)M9)i{2DbO2bH&Q{H5luz>{aK`tC68LgBZ6-e-~S(5p?Ei$L5&kb0gy zr|KhJY!@R)jB0CiRuE7iDO^OgT)ca?_r6n2ADH98>o9YRL2>ZIoc4c%f`ToZCJGq) z-`h#TX`XY~DM3JyWy}w+Tu&-)d@GyDy0Tg|0o{^Frb|TR@`!!|BuruFYtcNMv@4#c zG-x@#$+tg}fKi;0h@PcaNal_+VrKAOvmym>iDX>Mz(jB#BZGMuXtY5eYExO|_VFysDWIp6SlG z7Xrs=oK$p|B{V!>!*_(eQI3r8j`|Vb9>mI_tsEAyjjb~)YzY(5n^k8b-U^WD5dG)x z`iwh9n*p7wM=u1eKKI6imT!P+`sn~8un>5khcSDm$~Pmms6zK(m#8$I{%l5E{Xk3e zfX64_hx|hzaMheW(Z2Y>L!TWR7?%wGr<~tX4g1OakI7YEZJXst{n*(Tp^L@K9f{Z; zh~DF595-1w;&lAxelIu7Y=PkJkqkJcp z%}!~IU&}rY2zG-cLq36OW88S%VJ7lTzxNdUEVCfb(yuZ`@NBvw?_jx-Z%yuv1N$0v zB^W&2ZtxH%#GIO{mPi_`sj^Ayky`MJAir2^G$TM56$2+ zjx+Ls#bDTh9#QKK{&`*_-|xVr-(d2doH~YzZ!8hHX|j$wYA*0%(Yu|n4vsvS(CNsH zJv)IM2VL6?n5%>PA#oWk$A8#)6SALOx~zAHh=3k~<1L<-l0VbIc5}YoZz`Q6PmmYs zoEEw*r}QzA5W)-OFa+sms+T^!@YA?keTHgU#kMV)18+p+0lbt~xX6q0c<1I_`45DJ z(^QxQJBnN4QkW4h(K@s+3v3ESNnArWNMtS$t>bVtzwllUGo8CjXP(E?TS9XD8G>W8_=RgEV&E82vnvqsE|h>Ob`aW`*>>g|v6rOm4!q+Rjy;>HtOyhD9$3 zqvSNj!jDXW(0(KrtZ7NMY;A@MmX#f3#{3L8_l8j%|0!gk*9Q}W@Iop6V{#N1h_jDoE~oKjrX`!1M5SD%nB!yqumMO4>j>-rf8L_+tN@`#>$q+mpq$ zAL3d0;6g9}z&i1rpJRSq_18YmrO`a{lQxULUoJ2ogVT-ow2XLa;X=5Empfjp-F-05 zf==p1xoM^!%K)7bk2@_>$F?zx9JQpzoW4hVtQ6Bq%3(AN+@U2X?dD9bMDAWaK0Wfv z>J9FYt(vOK@&}3x+%>0$CL);%;f;=~Z(AF`NPhFjwA~xmN{Az{GIqc^#B!=KHR?$V z2chtKuQ`e2fdta=T`M^&h!sIR$h1MhCZ-ZTw(zJ{X6KM1uq8a=g8-zX!7m_-^_Cpw zo_V&zLA}w~>*eH3(o`fL5eVG(<>RSC!ULjE$BU9DWvsoPI=yr{W)D9GZdC;>u-Ae9 z%rKS?Do?F8EMMp|9~9a;Dy5_}@Q?N-^fu02>Z9d9N3qD7)6azS@Xj2Jj*ui>qd+&n zcYn95FCPXQGsSq8(lyKpnQ8Y}f2rrpD)(IKRWO8heL(Ii$i+XOs@=xj3B5)%u(LDJ z<0>&xR@!nulTqe})LE|@Jqsm8KPB4QJ>~B?Fjg6&POQ;CEYMC5*|C{Qd~*uh^!uk=)VO2&n0cl=^$n*-atHQ` zXfT3uG`261$O|a5ZlsG`opIFSy?mZJ#m!D0IhxJyzO0c6Ztd!ug{=_lV7-x@PPZZD zVRWZbhz04@m_g~RWdnFjHn*Htw%0Xrsu$=GfE0!j$uSWQY=nE6`X%E>KCyF-BAN6% zFUlsAjmIF$H4L5V`QQ2`m|v_N5v4AGQgN3*!OV97O}=-!$VXs8u?Ft#vid+WmZzt#AuqxG> zn0$-6=R3EVAJSGA&PJviy1Z+0{Crh3aWF6&NHW?=!*PG1!U^n;J6~uHW$(Qy7{Si` zL!)>!S4yFggG=a}VEN*s;|gq*yh+QPYe8%LMzdjjfIuXSYJNYz8oc?Kq)vfX_|FNP z!|Ytf1RfOs5@4d`=@^0;Y3y3=Jtdc~UN@N)GZlv#EbG5d_7}D-;i`VFNZ&qiTAkr+ zH#PL_;?XI(qL_o$%WWE8xz&`0^Q_>t1rIKo^x zwCQc=yeFMhGF_b7ns zFRu+crxqIsuFZ^LD?>MM-W;O+)e zG30FNnjBs2?ZS{Nu0b4nAtlcgjN|{zgn8=J-NXYq-U*Ppkd5zFeRyI@+OiRphMdYQ zdxzns)w7=o#C4noGZ4@0J0+j%9O7?`X>vvQZp!xB!*JZhc9-rmwVVX5K^iBGEZv$| z=~AW{j#z@UWV>pNw40{fMT_Ac|GAcsCY80#6tU>IzV0&Cn;nXqQH{F~WP5ooNt>m8 zJ(_Od0R1qDf3iHwjF}WS>t0Dvx-5(0*BmeMie^n9{PH^}K_}Edb=yAofHHq5(HbXNWukle`f{B3k}!?KoL!(TBH!=<9T7 z5!{xoSNKhfnKmTl!;fi9i>fc(t);6>_t&)EJ?hJBH&sr*kYaa#$OnFszR>WE_L;U* z39nZgO$aC6lY6=lL?^ah!G*uihx*p1H?r;;N7y*)40oC=yLR(x)Ky9V2@#e#z}!v6 zJ~G_+cI`NV9I+fb?F!j|4ft#)j}*|66p5V3)86Q7AhlVG1e@aJX20*l+)vZoyB+zH zdqsuGh3=l7EW%je#h>9vdZUn`;>mu~3A4FwymjWw`;syJB<-;Tn~*XqLnH>qGtjnL zFB6oV;|8g_q-gzw@1JHvS;%Egm6#vq`qnbBgj`p&zoNAi_ps+o*voxYd;9gzX2hC+ z^iOW??UT~xWeSYalXGw^XT)b`8X z2@r43yeQ!V$27T0AMZzj%s=Ev?`^O-dDE;2)MYwG*;jxW-I#19WlgW#>e44aJI#3d z=*6PpQxQ3dSVFqW8bwkFSG>)d!R#TSarpwU!Ni|6ByJt&YhPgZmUCXF@&c2wwQJMo zzyo`o)l%+1;0J`0TzDQB4&WB+9G;nn33u6BcjmA=17NTsC9(LzTIwLkOGsTLQ%XK+ zh8F}%;c}vqG4z_37RSlgX5M5AoZ=J>ybI*cfwhrtFQXoOBU(rfuN(g}<3))HyUEYS z4t_V{3VL^gxsCQe2NxDyz8PY2iSfBDNa|}Jz#!pY+JdZN|NHBm@HxO7z43pcK;WOJ z{MqNvUaQ&yzd@t_3lRU`eC6`XJ<&K8By!%v;1QJE1CKrHM(!tGqH4Q*&Ru0;4=P%x z;iv7;3N_HP^8xk)(D;EP2zd+FOaJsgF#?=FtmmCsVK++t_rXK?;NJdB?2+9rZurfG z|7mrTvANR0aTnvH35q@E!vv1&oc*mn)Q_mZMZ51(tQp;gk&8=iwVjhxT8HQAd9CR3 zP6H-`&e_3X7Pp(AHxI+D_Ka0)K%vm|&dyHm$buHjfQbU?dg4x#RY2u@KXLx3wP5%Y zL*_4jJFAs;^5J!7{icH;s%UQWayvU<%)AFe;&);;H#arDOL(505^8*m_4UOOfei(D z&#qN-CPVUGOf@Mn!?-fAV%CE6CqMU^#e*P%u$?@a>48R+wp^M3!enLu1_J%nN}Gwf zLziwTsVAK|GnjQeH^6|-Fx{)xl9m%2$h@2dWEFR~Q~ZDlI2dSRJG9e2?aX%K(l@I1 zq)k=*eq?$o{ri>(&3`o1_v2WmKfdB3=#jadT@g!6C&OVw1qU!^|A%amA{tL9++W$Y zX)C@l(=NtB)kR~#-03@QGX*FPc!8%jlrWX?F_=_dl7y{INldgN*ap2;>-D?-2o(5E z-^^^Fj>})EjVAnFHEdzwkC5p*P3G)wkYmS>^T3L*_SgWOBupJweM3bS5oOubagB!DaF{7 zw~w?fM3-A^LeBsZ-r3plzUf!zgJQhr zJoS*dzx)^At`A^2K4-71rNc5KBaV`P0Yn(Ct3!;n7f~bO?T?@h1b%$f{lY$ zG{=qaCXK(DdQj`4I^8@usPgw#P}lS6*(ZjIwrw40a`3b9<@;Ttt+IT>goEWz?y7|9 zg>3byTT8~iHLa51*?>NKsAbkXr1boX*AA zUF^YC#(|QgIy+<3rSD(}4cLKuz3KOL4b2B>G`$)E%p-7H*E{#0N@#F9BJ9lkWCiY)uX@3MaAp3U-e1g@ty(Ub&WwgD7eA>nTUU0XTLohXSn%~+i_6pQ*Zq-eG zl5vFHGZJ34oS{)&tI#mwo>Z5BfZV^=zaE5eX1&qO-Jbo{1qw{*j~%4X~9c$sK5c6??6_nYC0MN3BGx zd1K?A#+*;L=Cc^sjs8qIB*JXQ5e~AxH^_pUhfR#}LiDJDL){d^w1o4v^@{BZOocnQ zA8RDaHRW6ovf)!uymtNNkmUaU`iQ-&s@-&{OBv^MmY%OE_^`uXGo-BP#jS)jV&r;D zktw&MSGA8rN7WGDlcS0s#%gtArR;TLrmw*8iblJA{#~Z)Q^l~;}3TDN0T}Mmb%mc6@@CJrlW1uOAA|;+rD0ujgqCwdC(wsY`RCB zHA{Pnz)Zz8>P(xuU#6OvY2<8mL6KIEXjPCx)4_*n&D(ka7W|l^6pbiGD_NQ ze)1M@OkXUnkc+QXUkrYriS<59s3p^;r=XkcO0>F;306JS47LqH>Zt>q#W^`+{K3IB ziYy*{1AdRO-DOD)!$^b}xoc^ydTdz)b-8Y$v~K#BrRia#TY&GB53IS0RD<)~aS%O= zL093;ArCaU2rg($qho1&OfXT;(`Xv-rR0(B;X|W06T|P4*S68J9w?G{hClW*rJBi; zGdp`Nq(vD6YL)a$L|k7z<>x?z|1zVl`PTQpmzs?5rW}WR0pZ$>b)o3ZVS=m2@FBg= zt&z86=f4BKl;s%_$x6DaZPNS4SI=K;h}y%ra_nk_ zOtIb}6>`K?U?AvOyo4?vwo@XY#u(O+w4Adr-By*3BbZxDqF!tE?C9p#1Rcm3^dBis zUu7tkwDW6N8Lti?uaNwSZEn*v^g9{{lISYzD{vwl*SdC%E7&?o|Ioo2UasM;o=K_p zO2R{v+c8>N$enwsaS3c!<6k2)8ek>`?A0(o+*CD{d*T;w*oN?-ess#mn7+F*QLXAQ!& z#x(x&ULD7unf2>a>MtMPEgxXdrqI&j7@5PZ8>T!PTLMQecuEYtbHx>M&3GR5=wQpF zOx2-w_*G_V56`e%i=!)RXm_e8Z(YQ|%vSzM+;LV~tT!ri<$GO9KJ<_nf)l<oO149Cm#QdIg&(g(tSU<=a7dx-M)zR z92uwPZhcgufG~q*V)p|gNJaB8jGN7Juk=dcHERdxYERCrR2n_%YPIS+rBVAFqJ^6N zvZOnV3D@LV%V(wGE-1_*Pl@861s46CHQb)DglCAF1_IxCQeN_qY>qzY9y!!aF{nrj z(U}bG;g4@u=Z6(DkesH|bL~n__Z6AXm{wcCZ-!&v*tU7Zb1ym(Rt!@#eUCfccygR_ z)+LM1zTdzl#7mB?p7!6w@AJ(R6?2W2*wu%IK9nn*w%=PMg^l2PZen#Rbo&CB>8hj8 zXuMt%9Og4@KNgBs(5g)tPQxOfjSZnz;@I;Ov;|EEFsH|Gdi1yY<6B^nVt&4(9A?>l zucW$@Gfz8j$plQV-9F|kriJUT#NGQM{De!*Vw){DSYVYp%N!?tZT1qb{pn{OBR=UT zp2MBo-4=RN?)OU)pI026Rl7$9Z45r8Vqn~QTZTK;zhL{#O*_z*iO{21w*1q;$au@l zev2#R9_o?C_2xV1R9mlwS4QbKXj2sJ+a|p}y{OA8!z#?LrK7z{NVK9~?tJq*$5NVU zfkZu)$hbcBzmbN5A!cf+W!Ei_JJoY~Hew0KME-)mx|bS?=70F%R}|@&UVq>C_B8Wu zQb>B3B-%=FTjLa2Trn)=f5-JlV!r!u28}GKB3NWf8d)W+^IC7daQA=^*(;} z$j`3iGFp8^>+z<+t5mRlIX^_BpbUt5uaNGmB!4Qu1We_^ia%hO65PNs)Y|$bR({@y zg`e;kJ&Dr?+oP&RM;o3aoqAf5cKEZFgzCqN0~>9k+q#c~xqFCl#4XTa-qygqdKrsP z%kCR+StX_cN87*J+xLW8M?CAa)LoAF}93dG36lqf?$QkzObovpgWp z)!ryZjBy=E^hsw^(1Z16pm(p9mg$KXAr zY&fH32`H^f z*R(E_8eU4QWCl*-IvBVY&s+pLsgJWom1&UT^iJi{Se*>-kY-uiNtGWT+&1!;5F@%V zEuQ0i1K842)`87d@39--@V7VOcOLvCvA z(S1+*>C}X32C)vT+dFf3G-3vcO!JobD*sk1omfWgFW zvGjX84^Lc*tN*w}%Dd|X2Q@<1j|#SbJ=6Z}PBq|r+~L!bD6j+iVYs7T^e{2eL|_3x ze~(Q{KiuR*Oud*14xc<)QK;3NtNRr&6=?+R6)f0R;P0w+Qw_C1_8Q-07Sgn~ZdfS# zBR$W;*e=**1*rC4cZKQqhKU7Nt}bI4?1(=+qz(VBmK$bbd-&TWdTojD(maXV?Sjv3 z!qB^s%IpamqCKJAN!;pQ(2Eu5ICJe3cYM4-+6S~>deF99 zd{>>W06j%pr5j|y$X8C?hj7#?Xu^6c4O)<%l5#k_hIO$iUcibslyd|kc>0G(BC1hW z7yV^2j!OaaqBYvlVEvA+Zo0a4U8)&rsGeTkGK|y;952~YK7~jc`nDu`-9d-VO5$d} zS{Kj!%P$Mq%o}3Y=_FpW^T!P9PQ4RE_RbCt2Zr;hbCCRgM{=AjiH1^!m^D8dq9kzK zR&nQ;fhw#;($D_LcPRmaavJg|-LkR#T`M}s(_?r%{DWiYc1%@l?ShSuD!L_fc$XAy z?et7O}SBP<6(RynBIMQVD zU5(PJkbP5lO)@%K;IhgZheWpsJ$&--Cp1Ss6k4gMiNzvqgudMkdmZ(ujl{eHm^QKh9M z{O}nGo;)hXy+^#rTKunGoDHf#EfOOA-D|I5y9VEW!*2ketUvnO&(jc#Z{>Cf&@$t1~s%^d0?C-71_uQYmga;)4?y-{T zpUINiE|_07PN`m8t;uGPleljJ>iarNM2>&+qgN0_Xf#nL`2uOHCBOi{u`t(LFrFbj zTI3|;BQVKo?b}x?1i8s|7P20c`N=db?r_SU!S{Y+$QQT4&uDVfhlamxvBCCBJFbhVO)-s)Ma8Ws(Ih&53`2l6>P$Us6J6{8BGnD?aQAkO27m; z+XXKSz8Sl3xZ+!ogcy0}BedT7VF#N3v(rOW5(GHx271MyU-JiYO^mg}N;i`NGWt$bK=Rj@14kj$0bx73}wa_c6g6a%w4T)ZJp!UodO`|q zuiQIPt0q~Zj2@b|`jF>R9G+G)Wt2{Svqbz3urr*WYbm)<1~noGWG#VkEs zD98-G^WD@JYX<)#*)CWc@s@4T(mnx`j>4m>rxu=RcjXFx+JtKDsO_mqI9DXyLGe~# z4{?~4i<`y*7>Kb=g)zpdzl5Yqsnmdu3lHFxLPjGGh5-9GYP{q}e~??q-SBk8Tt0X` zWhx^S+PD{!eaG7iV7mMxKc1^alPO`csM)L3pX4#k_OQbqTh{lJWlnfp!b`b*Xa-KG zM@-iF4;AuIv1zpR-C|ao=sT2VMRCE>tEV~!0)nK3P9M;7B~ztj8SaEZ&xS56o}~~z zJjTaZH5g;w|MeprepF}H%EP}yBd80b($IG#75i+h#(u$DUyDgjKu_}8f``41bYO2}rKh9(AVse#N z?@<$aU@!9l?WG~<3cc1lW{0ze*;=w)BHs?X;8{fbabM->_sl-!vxgV$j^WkW2n51O zK)scBp=3aR`Dddb9kOun!c*1jB|nrKR{L>;2GM4>N|yHVhLXX1bbe`mBpY&0-uj&P z!d-Zb`s{}y_nxBO-00aic+*kdoHVAg6hAwE-`lY@qK!GG+c9rlS__T*#mDMS!Vi?w zIW~j{L|5fTaoY{;+DgNpUru39#*BFDgbN${S$YXo1+jzX0Q=hhGo@BONF1b1SGAW3 z%k!UyLSpjDjBZ3HR$w@g#YVE_dE42)lB;FxO3LvFlw>y|baE*3*YWb(4=4NURF?lb z-HtnJI-YHCoR{THRyd)35005Iz|qIi@VkN#`-CF^-qse+H9B`lHraveY#H=Ah~QUz zuQrkcuUITwzL-AstN$u0<9hZb`?(Z)prx8XUP`X^g+))~rrlV;*YlrbT}V@(K7}Xs za@yPdjB+(;5ZpUUN8@LYlxh;(*oHXF?%nlAHh$?3b1eP-cVPKUyTUN!<@%y*X7W8z zap`lLsXoYjE8%z`?Qd`0n#6sp`8eh##nbXt!FX&p{5!yaWS7%A*#Fmn*d~dGw@Hl% ze1HgyoaDOZ@c0c_s{ufaiP<9$NA>)?WQhw?ixYOLEX`=g35!J+*xQL9db@e(w@a!Z z3KySW>-pV%EADneof8x5mFxN3C}=v)H?2;O4?AaYmWF9?UnJRK^~$4^^)*Rw<_cvK zcC+IsJATIscWHxuaPM6RfKk?7q5-*&mJf`TBLWOw9Rxj{@8XEja92c@FT)Z1rh<7_5V!W5(;^I&_EMSp4ZdkKxWD-z}}0&(HkjWQ`5D z>S)7wGj9y&n6e=HypkeYB za`@Tdb8N#FeD#yL)|wtu(;O<=4$_Q~NB5MRJqFsd#h>kK&?s37YR+E-1TXycR`yJ= zqlc4h)~h!qW#?@bpkavwj;8k9h;{kfDgjR%)4yWXjM%GI$xRyCrYzDTxPR0Z%0UEQ7b8-tc&-T5c}IbLYck_ww7B9um{P zKhDlHNFQ@a6%hyrzRgJ~d8G$+Tkiz#`XHJY`nXs6wd}ls0R5(TYHbR{W`gEnqLIp+ zrohiPzS#XX$=p+%Ph-93qQzs zy|07$rAj`aJp{H*&*m~;7+JiS8--#vIk*5uDb#+Fy1Y_`E;KJM&k9{gZ67Z4=yW;g zpKb;=zE3$eUE?{GLE$WK^1y(%{gWR5t>Cw8g_0??s4K2C5mrk37C%x#AoM$en7{nZ z01IMo_vpKSvoWU3j_50xv4E(Rbd!?YogYFGdlNic>x^y05zjxRqDhCY_8%>B3bJ^5 z{Y#Idq^xb??88->%Kg){Tg+pZ$+p~%XOJ&QcY)2phG4lQU2$3(m$v0 zqOQBni~=Zs7nvf~3ntW-*pKL9{!Sn^S!hrLI*4$)9}ZO_yD(D>eL{e-D*Nq(DVUI9 zEzasU4wGV(BjP#+s<5-nL}-!zagFIJrz_f1U#A_gIFN2e99FQ98Q;6Yb0o}M-L{{g zA6|Xr&5@tI-s4Y83se$*t94@8_)cO{TMK3H1pltDDb#BPTYq_v8SuO5nZMbcmLvqk7+c4Ib|s}J zhZS0CGIy!B&1jJNvMoxXVLQYfEo6Ci>LS;6jGsT#4qHXa^CT;2=Z##HsCh&I-1ZUH65)9W!J}-8uZFn=KMwc1DxWbL3gm+6t2vwxiR@SP-#n^%1P&V13feq!T)+0>oS`OG&Y} z83`z{Ly?D=ZdAAbQmg_1kjf1g3-{|eO3RfwaN6?cpuR})^C0ev(;e21XC+Q!_zdUrLAd#3oh%Da)&HMH6KO%b{MD=#^%1l2s+5Hr`nND84*j4tp_W-rs zA@YQ_*}Ji-A6-4x>+<;e{NBAxsd^xEt=vD32$iVyka}5mcFk>Ja_thOaww&jt9B=wo zD`~TnKjfJ>1I1QoAKJMf+0+h*34(5{We$EnX5DP8AG zH!=ijcVyW>Wp5foeuU+(lTZ_MG!63?a&-PY#iZYj`-;{Z9^am7?~rtQ(GEW{o-e6I zz55u=!KfAGYMe&1+(CS8nX|0ucMcIE1{bv@PL~ZW-#kbjU|3Dd%7hlETer4w;JxEF z`!?^6sivT=dkNVdxrh zH3ef5WQ(UZVwjE;3?c35lHg7Wg_i`(*K+4=@_ zSi(Ynu~AUq;P~TC9Z!6_U^xeu-Q)kXt$7_FwOwQxF$0-q(2v19f{ij1@4|prIGtXF zbuYg^ZVxXiq6PdK!ZfxI>%{^pqc1ze?ZP|V!7{W@r<@f5D*VpCWhxm0^?)ByA?>4=z%^o^z?R4v(sE_} zVzylHMQzj5D<_WwnSeTZ<%I7({KB3gW|smY@0bY9@?LLcTK=SO)B0zyA(EhwRoSoo*BMT-d5YLnF3C^qoAa^Wu8#Ym8y#l0- zZS$S4hwR*|liZ!kq4JvTw=K;E+i3m!Xgw#U@s>Yg*>>hyl>*)^& z0tp%J#PHG)yR+BLJR)G%ZwBwd1O57N)D@M`sl-12RX_Y+%`C*0E2r$vH8F%l@}8KVivyuiAh#Xig4!XLGw5 z8-VrWd#pV_uz|dhEok$Av5Ct%FNa1IG=c{5~qrf%{Gk3=KEBtiJDu@V%#P z7wqTmwhi24!2E%6hYgv%SsqLvF)A8Vcj>h~PrHySE)mGT7x$W3 z0j(|kLt^6KuV!WsF^%f(>{PM_+wx{pfO`~HUaY4U9H;&d#be&|PwJjy^rXAG2Ic2HjaVlD5R6BwMb zGl(=8YVh5(8pK6W`aK-|GOr4|%49w}z3C5%IyG?TQVht4fL zReIFYI*gV14`8@63=CaAg*n%B(HVpq#oX0#i0=!vwHPapCjOTJe^4dkLk^|jqkeAY z-rY!iyD+o4O=p7;Uv2}9RsFw^o-#RHCvTR?ojlpMniTw*z~+>(!i4L@ggy{Ughz<RRmcVTbfc}~s zr_LG7%I0I|saK*fd~W;H%(bee!QaDPKC)vsvdCelKtU0-rYTESvhfq}7uBN!|?*{+l+&J6=mo><rrTLch0LvJ!ewthJ7P=qWQA&fqUfq^6dY(U=Zz;uTwjvb35MC0`FVg5mpWUwG zPJ^Z*N6b4H_!1v4WCV`8_+8+otI=Q23PGH+b?z_Wl-t8(DF%3jm4Mq*zbn_*jD6`& zhpM`lOmk`gI<&31=kF|VyDL*o)O7ZZ0j2a!Mpo6Q(DQa^HXjmGbsE4%B|QOL97+8~ z@3>d>)0?~dy=lIq*RlwoDBa7&&5y9WC9_t~BLO!a==(fAM9m~thMZh}d@^pJFSBhiT2eOn&Ty+l8U0I%Udo{l!b{zYd#b}FE}LUq~R)YLuK)U7o0Sbw9o;8>D9iFthnu{A|SCe zx)8|p-7NnQ^-M!ky+MNF3%6*JmXhScxuU;QmpHYY6`VZZfS+8MGdbKal1Rb-Ur(O=U{eVd#yX6Gl#8W`idJ~NR?kpG~)dp_>9czoNOBsr-Z*sniT>=M5 zxcf;6v!7Sy837c6oN-#atTEn;`wS?FYKcH@8xr=n$0j0is@!Kgog+l!{UqM#A$~5= zB!$kNBLuF{G7)}LQ12}8#&gCIgmXCfkMyb%n{sOaw=EftlWpiT{ z%*3rn0ClG?%?zvj%$ghIs~fMQ+q)}OE`5Y`9XM(O2mR)zTHC&pFv6YtF~v4Y`*Os`||YR&EUM3PZtf?y_p6mH|g>;9vd8IQelyiJ6&5qy@S5 zQxbNxNDE4BOOyR8t!Or7*TvF_5wMNg2;y|FuDfz1elqzb&tPQ4JIz|(W`BN3kgSPX zH~p7eU?KGkY2N@RlD>ZQ*Meg|FI1*8@4Qhsw^G|8LfV!w``77um)r-I3_-wQaBkTwZ;EcBYQ4LR)jn-;;xW3t&*$8^ zfb+P(jCo<{w5tB@s(p9^eXex}DGM!=ebjX8Vqa`Go8MNCk3U9g!8ah~W3Jp;cY5BJ z-u~OlBUcu$Tlig(-L>5AO}9MRWr1&n4!`A>_JA#+k1~iof>_S{2a*j@x|fw2-g1V7 z2$&8NmS|EBg;UmVG53924)k}|bd55i-?tG8UGDG)vKes+@7^UaRs-}XM;z%n9PdC$ znJ8=OprkMKM9hizY;A7Y8Qv~^nTqOe^z}>6Z03)x$Op+C6q5BW4k_KX`Pi_xFxV`) z^O{D%*FQF~1g=Qhum_?%Qc<^m&aS^?$n?A1JZ6W)B`2OWb^LIdG53?Iy^!<@4vyHKM`u zdt-^;;rkiT5z~Oxk&67xW6Z!&CY&rKpn)aVLWds-I(|=uFHP+i$w`_urDu{mRM#=G zXTyIEx|%E1j*Aw2?g#wyHdV3rHjXRjy%26=8g(G2=U^lAWu*zZRJEVGfNMO%XP$il zj|h_bd#MlEe^(x}$r|}j*o7F#JRi80{VR#pAC%QFPw4k%(|E*(y65M4uvwddTq$`H z8K7ASSFP-SZ~J<#U!DJE&PTF|l9HT%Vd0{(`3>1Utl6st^4~=8rpo%9bJN=`6nRY~ zb>{3Ci%dz9`&~Kpq__K@uZyCH*}$(*Uk+A#wjSxeIt|j0{dy=2O_)A9o5`WsMO0PE zIAqBAd?@L|jl{Gj&ErDy&RoyDsV)S&DmAbt%H&AzFOW{If#->TY{V3TZ)E=#N5@tW zad~IAW%q~`c>^*src7Z|?5x3Sk`IzAFpCN-SGwP2zGYi8VvP&Edz7R0O(EDd`irYE zR(4lyx`fAjb;VVC2Z;e^&fF1!=kP=-cFLh%37fbRmZQ5*umZhGONlU>{f5s|oUFm` zQm1CpB)uz^^Vc4KFs%hTKjx7B9 zo3bp)&{CGVJ8SBln5fIH1>2Qw5Tl;<>B2$nUtM!7@c`WIw0GcXRv9n5dpEW0Z_cU6 zL!m2jKDD3BA_4CB=O?N1J>XVlJaRHRFHfXamu#(99(8hiEGou zBP&+j+`I`YB&HY!t=^S^oG$+XJ?U3nVcSF!Sp-^kjge|<5iT@b&e-h~%nMe3+1(5% zQ7i4Vt7ay6;PZCN_VBk&&)sZ0FZyWfg@fyfJN2eZjBQKdZHr4t?}N`G-5RGS*Jh$u z+D}ir&6;=pM=Q@N5(5@G2oF9|CVgU!?$mC%?jGL~Vs$^%s{KlL(q;KzVew-jx=&V5 zQ)N#6<@O45ay>YyD59hs60s=I{b~?zlfi40a$Ra%FJ=!y*oRWnM!Ia7iv!8gp!ro+ ze_Qq?#z8k5R5*1Z!s_|lRXy%C@}RSGzucC-nM@aXmU|Y~wR9u;E>HKrJ4W#*YKUMu zyu7#6^-w7D?C+R&jIhOdi1lZI>!CY9sbKSW2loA$W)vWF$q4rytQNzC}PnLdl zv0eJVUThbiqQ|X-9h_=B^>>aPS#LhT2rb^Ve)?4>W>2yK{eP79o?%UGUDt3B6+ARO z9zYR9MFFK3r3eHYh%^xdr59-;T}o&{QBbLZ^rq5PK#DXWp^Eep0jZ%!YJfmO5|X?t zc#7Y1-{<+>>w5D;cM`Jp+H0bu<*!NfoR)Q#1$0_-8;0G-!*q-F(# zrj+_o@*CAB6JWScCat7JsR`!bAoz*8gQOMsi6Z?ccxxM#<+F~2#WpscVT8sDiKX+pRKMf!fbsy|&ks9eHBUfIOBy=>WK51~T z?ctmJQavM(IZ=%>P5S>b}{xgJI;=SW*DyN}d`0Yi>$M{uqjUtiPp+EK}Y~{l=e(EoMr& zF2A3f-maOTT~*~T@J02hqORWDc~feU!u$fhncJb>&EL8i@PPwLSm!=0o9xC~;pyMV z;J#)xc%T)+K8WR-d+acG-X*UDzP35I)5HF4G!y1dY4n#u>jX^?SyV~Vh60H%gM201 zb>R=4g3`zO_cuy?mku(*ESvBKAgCfyn^<%=Q>Od^dmM)f@wh7!e~NgR4ShD2cw!?* zYxHa(`1*#09*PPy$`QBfLVf~CG zetzKn&l>vIvKQJ^`Le(I>8mI~NYWL>%X=Y5tWJt4y-dv1?)R_KWh&T)c{2;OB0H8?M#FzZv#+*s(otH z`!HIC^g~T4L*Cu`B!mAcUK?ZjLjUf~B`|%1zBM0ChZispU;?tx?w53P(rxxH3OVPu z`hZXnyDG4qK^;9-0|_3y2pom21Hoz@5iXU?Y<>rVa~ID*Ao8cxvsEBnb4S=9^;Z3d zPJt5tbpuqdAHA9$S_gu^xPw9j4uDToW+EuXFb7xqH~ClT6gi(7K4L$tAj&S*KY$k(cGNp;tQ%v+%?zbuv~ER zF}phJIkIx}Jcisw<^8^LAhrW0kZhlzF>iI8vz16T$e-`;>+%dx%&<$s62?-(v2tY3 z6IntAu^leA-lQC+%x(afn0FNOvkW!^=dsu%eMGMM6RT;LjFppZ&Wcv? z6U^PKI3!9dmnCaiT)4KL(9oksz{(yMqeQYNA5G+WKOE7X$75d=#|yzo;bf zX+R)imGoHEuLCPL)x}s7A{z_HBEFY@m;38rrH`k7T8;iL*sKa{d0G`dp3svCg)g&16FqyKVAYevLbsRTKT&q+myfRiN`Z7o0k-V@%6x^`yg%{RB*5WsQ2mT$?4ll6qp zFfbmvRt|v9!O2;rlVXM4UexS|`t`&+7OQM57@>4`a2RqAlE#;qM}UuVE+i*j=ahQQ5p$oc-xyk;c#;oE85+*tI}oKVyh~ zo=#ylm+;TMSLh>PjMtX-{z@Pv_;zbOm4%4UuR3{4przRcDo{n51BZ(U< zqg3A#M6RQaWz03zkMssSU)Aym^pc@NZ95LJnkk(Ge>{kgtrbB1#3;SRN*3e^xCyU- z=bXcLCHev+Makn2l&YI!)1=I-V-2IWJ|AkG3wRKPYk?D%Buss=0-3N+7MFh0Tj0S| zQM@b=-!dD8f@q&MtMghfs`Hj#02}-FGHE-zPm0)e=YQe8wfuRquSOek$V$iCma z#a+QD|Lq1&840%LRD4H@7PYIj2zQJ8M8KEUeG)N|Ynn(I#Uz!hb=w%QKS;#UCn`78 z;lA5$EpTrh&BorHC$rUG@*|i?Xk%%kMw^+fQbg=dW7((H`TRR`lB8WgEb>WjVyi-v z9IXcn0Nzbb=%T2oXxa^bf@;dgt*xyIz?Y}v-{|jW9uk_TklC(u1vs3jzL!7@V*F^L zC*@}ZlfaXMC@m$f+~NLJPym!G-?gj<-g$bC4n}L-bbJY;YDVv-y(<8s2(RoAM`MqDVt9+o;UeN0w@{Rs8koh~;OwG5R}3~JhbvS0ty&(`il1S}1Qy|4+` z8Y~@i+L3>|&$vjmF6)D;?Lcsqw!rM{q@|VB)bnqTN9nw0)R{*O*8-4M0cyHQkFSg4 z9tqFRZp>TwdM|q!w3bKDN*B!Fl4og!>GlT6%4_Pf8@tMi8WK$Qhp^8}#clULz`>%6XjZmN$cZR8} z;BFure37IHr|U)@>dPePO%j9nMFAWKF#gf3$LBa9-H8nR z0^k+Rbq-&6PWjDsr-bIG&~*sy`ucf$)I{TD>-#~}!xDKu$d%HdeP=@#18J3Jef-My zLVE5mY$y6%t-NO&e<--8(Z1&S9EUNnKgJR^!Lm7Oovorkw2}@zlXy}naV9^?k-yD0 zX=t`Ie^R3Pm7rzpQi?hP&D0^q>C4=O{J=owtl*ECQH6aF{;NFGt^T zv&UlPOdm+y%Dl1B^y;cdvdDFXYF_msc@1$#X*71hJz{@~;5l_PF1*KVo5+ZEx20Vu zOHY7h^In$od2%cw<_c;%sHugu+Bf*mPD)${ObT32q^#!T!}mc8J{6TG$bY&XUkJx4 z4)gGyXggEZNjPSSxGGZi11<>u<*!@Vc!^U#krpz&=EY2d{b%w=7B&L^XB9)weoR;yT7IrOn0);wby&?_fomM%)AQnVq1U7q`;8)ZF}W z5=yCGI9svfo&1<*NtUJvwA=Q)qU^I-4>=Eyqrl!UMEcb;i(Nx5x$(eKUS8*8JC5uT zye@t;3@ayTZ+IA@{VBzv@F@uRmcwyfz=zAduy-7gjy-a}doK0B7b9hqYu{iA0jYQb z`2?5;e0i~$)D70M2ZKoTvTnKT9c_s0!2Lat;42>4ReoOndtSf%=~$@g0_5@UT9L+B z(06~__!YO#0mfZ7Uf1Y%NaXa9ahLFac(h;OkVYn5fOA(Cp#I2(TZ)O6YG7bBF#`tg z%f}e5jO(2RkzuF^93{iPj7G78v!9lYF_i`=k72%v`1I*09}c6t)72v(I-YEo2A(S= zaTiU3CCnp3V?Nn77UmE$+TnP=n=<>UZtH2wu@^Kl#~Hf`yM2B(V$}KdN?kgtMDR8e z{1*W`in*=`d1)%{@<0g@)8E%T_AI22ly(}2K;!Jq5$IknA)T~4+DxxSP0=Zq`-wp^ zVoY8Uc8UOs-^+L&`zPL{+vz`F^(}xRE)lZ10g;WOBf?$CSW?ob@e?6}a3UlH3>Zjo z)5Bl9uogN%ym^H#X7E4N60V5vffV|Hz{JzNsscZQ&;NPXzdj*vd=-d(X#MNWeHL;O zRqm>!FW#BdRq-hIZ2QufY%SCJfCg3s3ybrXVZqxe`jUo;+JTz9nZdep8G<7qgbM8#rAd_zsiV) zjEjuq8YM%Bg=Hb3;{}ok6{q44sL7v7wg=fjozlRy%hQzOS{-$Z$W&LCs21&BU+_(_ zRZmzI(QyOj$NaF>^jxFZS#Y5kVmf<(;$Ioi_r#s;c7CoffE1QQ0X!Ul#0)OOk;9pc z2iQ&}>z>oc2@B!M#A0D+EbLVr3}TmwQvyRj^(DO%b@s^oECaC17h`&Zs|A0qRS-!O zYc(^Ze4??j(RjD_gLoVSyuXMh>L|k*WncZ@;`#s3Q#Ho+sq>pfon`2i2i~{$X$Bx# zn7rYBo}pJ3M4c*xe>zpb*U&Eq(WAc{5jFY<@tlo)j}#ZHTkSdk`SM1W5fZ%r^D-Ou zx-02DO>|{|C6ku3xrwEQP;o;()S#MqCo~#PwMTCG^mkZnH{ThBt+e#y;3TM*{I-VC z+7`Svk3%;tSEmbhh<>2A#kFk`{VFR3HoJqoiO5Fe(EQ{@)Zx zgM>T*qrgBK@gvOq{VUG5%OBll7t8fb_rh{+B^>?3_BHeV(w0+}Rpr82wW}ReL)u_y z{tqBhrW@|OYQS~W)W{q?XF*y$)x14VCB04K1Y)InhL0)RA)d5zbsHm|+;vn^2clHC zI6!}W`}V?jqa7rPp@D{3cXZv~x#q7_R9z11CXqY1_|_`P3EFBJQ*%2I7nl-7`g56!!*GNQF>eT19)% zZ38@d3r^0nt)9c?LgDLW3o^7Rt(96@k?{L(@kVc#H{}o{+PH=K6OK!6k_pKmL=ck9 z&IY15Ds4)=cDvIX?=yQiHjg~_QY2jZ`dAI@Hn@F}ud~G4xmnybm{7Y+Nw;PZl@zq3 zl@uzRsk8y#NnDt(q~P8Lc-*deBRs^)!WR16qq`}SG)h^UVlxVGMn2eV?V;Myyh1&9 zi4nS9O-!d;;=SWZNgc@L@s_6%K7>s5;nVFm{N0Q0;5RlNUpLq$@bElYR1xLkcCbu& z`vLb2MLQR$4HP3;j_G|lYDhgQ=rRdITrIdi`1X)*4MVkG{-W5@TQd)=jMikEyB$JBw^gwkP#y`Aq`n(v%S zM&c!CS=92Xa22-w*Q7IDH9?3CdK$-@oz@wa)`>OLnJ)u^Iu3b@D=F$P!cY3YbRB2a zQ}J-x!}P|ufq$?KCr_PZp)EGzDoy?MAKyJg?=HsybMc@6PkFd-yU9dJ4JPC>o0b<8 z-cd{}*|bI14%hmFB!HcT7?oVh{-XB=YXq566u0@9v3K#@DF?PLg_kz+?IQMJJ zE3g6`ui}d}u*Rr+6Ar&YRt{I*h!QY1jJj`jl%hr<(x}Qd#To`|Z+ z9uYN}9X*%D8%NfE@f+{DCBsKOaB;s1EcUxi^=>!K!|m z-dB$uJ2u|@=*5ko3hKye$4U3~nFTb3$ua8=zJ2H?;S`bY!#ry=6U~H#coQ-U+fnGz8IkAx@w{p8Y zA)_&~Cx^<^gFq zQF9M%sPD6br~HeAx$AbG>YjFZ{9c^W&cdy5`N=-l+Ks}6LW`O-apltCM!7v9>eOhO zbi)ULx?|+en&UoU6FX%pr|%tm^au?7P$P5L9I3haYITnBo8lJ*nvQt&o5f|H9?M)?md z=FqlZ20^uZ?-i%P#RJ#DvIh-Xt?cLjfQ%FSEZpKB)S}VF9=TOA$fD3 zK7^>yUdlZ{AzO>IYpKIZzUSvHOu=_a|B>1impq8)CW1_$c z40Z$l1&G1_!Na&8S<&LHuOJS*z-hi13xSMawTM+I5C@*0(Z|0a7~q1h{{_z<-eJE7 z7Dg1DaX@xY-eE{2NQraVwW|y8=;MFF^wC3qQTV@9<(Esb zYX28h7ZZd8<4wecASw0=0OSw8{hRaG>g%^u&;ZrC5552af$L)#^kV+2rrOf=JrIX# zK*`Pgr2Od}rq@pn1OfhpxGR5$kbbGkFPE>`{#WmXukCpcz{lN7;Gw6FuzjW*)}?s{ zxSt^3*c0R%caun@UJ>(aQ{F3oCLDkH?+M2sZDLeS6kdBLNQ?~~wCpd$HgqnkZ61*nN&{n6y(9}q}%Pmz3Pni#$$HPmUDkh?&`;Bj>tY&59c9F|S2 zf=R>H9=BK`GC%&OYt6T zZb{{`uj9c4p=uH-+&X4n8`M|m*?zv>%J@dFG|<%s>s)1;y!%_icbn)yX$FXFeSZBU z;62|vIXUT$`OZY@iiwJj-3rvN-Y+i-)GKd$U!Tt+PFzc;I`%U`WE#Ts zPB+W2GGWZq8^k&dpT!G#@G9^#qwy@iVQ_J~)F{+Jm88 zCPqbPGidX}W%m3!ki+x%y*#!chdwGLQ7E>P!>QjW?H*PO0y%uk^i>ry>9}b75e`n< z=$V;UHZNehxAW5HB|6Sma^Z?NM-|PK;I4a;9t#}c$(ElAd?3*lu!W(AF<4XObUwDc zdZ$eSM+T%rAcEP#PH7^?6Hu=>M`8KCzQPPzFM$D{-$VHR_#>X=)SU^Xsk%#>jyfi% z$OhSd+p%?X6E{A8^}3!5h&%-|TxR6C2JF+BRbwK|kfKVb1j0m(FAE0m|DC7T4>Zi~ zE10o72K@CEFObEaR!9v1T4sE@5VlD?AQI&7(J|BBUJxEL)&)gDB>8^5qW=MyxQ2x2 zC4zwTNt3&G7tZ`k@q}?7R|B{TY$Mx4v6uPJsIv;2gEC!Py#67bznNUy`dt;c;@~PK zT?y9ja8p$PD1_~+qjMVIRo~ES#S50j()jrJ)%qTw7s9YRg@ru;BSU_a-J%e8(-HqJ zACS%8>NQ2c4WH~U&j5oHVyXD+75#@GV9Hj_EW0cJ$UfZO-hO|N!-+Qxg+hO=A@Zs$ zAl5E#0D5cB@L~_p?i*U)+Y504JX!FS3c&1WL)cFLQt5vlWcLq1uqkl2#6K-y{x8M* zpaThNTNuFQp37s5{Fp?4y!$^;!Gm#hjZH23X($kO3ke3|XQz+%GXK*P!X0T*T>5zdT64Ha4@Ktp za~0j$^LI5;$=(;F&I1wGJkWmiSGC^eGz7Mk1F<+R)D=+J_rd|tv*MW9Y8*YkUVH>J zI+q(jU0ij;mK+q)!G0?#<-3SH_`W0?G8tlJlhS#L`8?~ z{_%tgybS#K_{Nql7m9MeqZt$EDFsA(3b*;DC%H0L{88JPKItyi6oaCrv3m0QT|SW^ zubPdEv(6mtmvko34Rc{#{Ah*)au7+mUwYZXq^M9~Sefv9Z(Dh9X#JgWPyoWAoJqw#RfiKFk<&d8D^jWQb=%>zbqs#&os_ z`Qn~-mJ{&?=>ExycIl4zp+wY=gaabyM;Ef|0w=tpatnLz*-qBYwUZR%i8$|7UTGRC zo_H~y$b=fz|2$>{@+IAOM{3@~k14%=RcAf!!W9?-$wxh;&cWQD!oG1%bOv}&Ot0NExEv&IsmlzMrtnyI@`7zm4VH6w<*gHstd0lDwfE*GhZi=P%Yu? zwc}e`+PS{*X|CM!>!J9$feoUIv0jAD8V+%OWd~-|f|l`V^gQPbb)P+$GjiX`jk@?T z)eA@HA4Jb09ubZ1&+}`Z#tochfJheU@F2I`US-bg(KXFij9Zl06ivjA-_n64&&e8l zRj)pqQ*w&H1Xlwsq zJU%|fOEi!2Nml-XHS$9LHtji<|9iB)KS|#(bYf<)2hVpHgYg*ussHDo8%;@OvJ?0Tw#40sZ8m2>`k>TG~x&k-=T|lVEmdh zcQs4`MaPmW@LSt4M4+o#u6U=T%F}pr3D{ zUA4~0IZbFxMf--7veZ3`?(D*ZmZ982!^VNZuhq!Hryj;4=+AXK)y7ma@x->dF()Vd zN$kw)oe}>I^m}6tl3O*uF6(fQj3cJq_tUbO^VY#=1>U2Bd$tWmCiZN1^UXu!LMx9P zUp^eEz-bZF@bJUCxQf^hR)!q}Y%$FF%u4rtp1^pL!#mgE7%!Ba=bJk!)TS@+&6+_7 zA#dD*%|gQ@^R^CEjV~q67arr&finR9np3epLz0&@UjxiiOaaHF6v{MjPjz&Vx?` zHMeXxl(0<&h=~n1Tk^~u4HL1@M3mw8kNV+LNmxX=C(Gm*&7|B;KE8tFkFxd+U9tVz z65w`MlT6S8mF@}wazxo(^V%YgWO)BT`IbL>jrfYT%Cly_tZPkd3zI3>SkC&sTXcFY!?e;qr&$0{!%Lh;Lu zF0yNJEi@`1_3}5bjRD%C^ZpP&$0g4i&j@0sFd7<Z88uppVI zAav$nE@yhhETSbSRoa2p6ZvZvsooL^W;&@%Wyes4#l=lUFR(eW&>Je>mviH) za~O4e$?M;qer&X={yw$h9JQ|e=xL=GZwvf#q_0G=lbkJ42`lFKFf8ozU5xR_Ih$yd zwY*B6FcJ*IPZ0C77;JWz-;@pxw%k6A)#)eoAKQj~ zN_)G5)=sr0=BE{BcX=$GmN53}pSX6^*I?X1j{IHYvy)V9n)lEiUj=7Mkz8J(4sWu| zI=^G}{1zBBIdj<~ge5FlMhUScjO3W?d?gVE6gDn;nnicpmWR#DG) zrHBTaX_g{s+pWuAqyS;lbWvoqc>j@f*#O^Wm$^EbD+xzGSzM%ytF1Kk$p&0caN$vY z(qU;02q@}ubj}o#5Ct;EbwCcCEqj#>MrWitWL2prKd@ja+x5IQ!V{-sYa!TosxmG# z(HVy=S6U-^j8to-V+y?Y43Aq?Tk%P+w#LmV8llz{bq5PNEehp$`D~~%3)AStqXt8V ztmT}8tfn<@7Osiq6~58`GN6%4Ti+mMeE5LF%igUkAD*SHdzjdFVAh{T&pLgQc6G6X z%Q}079_eU?=f{`F&$3m>S1CK2X~Rp$CIaU?Dzn!wGCY@@ZYbw`%Uig^?lgy6#CTs| zS4I5LZ@ zN)@bG>GRoR92JmmuPB*|o%~3P@`L7@qaDjl^A_5vURm2_GkYd)SQkCKMOizo09%Zx z-ie#T0lm;SThm)ET^{cp;RUnlbEUfM4~mV?@{jclJc4D459vD}kX-ffiXW##nG-oH zO{Hl^eFHz)Q3!b?EB=d)8C_os5pMyu-Z(=#CCMG-oN^msaTvh&{+L2;DOly?`uuCb!JO_ z&F&HnYO1!Jy`Z6i6H=}&TQ`mLkOGkw8Q5>jP5E3+9gYvfr&eQ#?~!o1fg!$RMs*ba zhNTc&@lI4$T!kXP5;HL)$Ng3{kWlQQ&Abzrdv15j-wZC~Oi`~Z+Ax;m8^Z+V(}^H% zfG(w{EtjVB;&C71DlDG7cwz0bkNl9w{G^38>Hyn2+^k<)dp(u)LY#%hsKjaZZVMw~ zPluA(UBUOEY`O_cH4@JI*t||&m5XF&yES4C11Rwmk4maHqVDy()yMa$%IuWW+jm45 zaX^1gg4GaXaXbkccGg8d$uqs2IJY2}k{c(^&Nkov3b=y77nZWUD~_v_H3f7mOeeGu zxn{C#x-+kY>2&R=GtEOyaMZ<^Eib3RA?=G{`(cPypOVZFot&vl*KcrF9#M&-58gbY z=fSer;4cf=3LIcT%pbrHHf7Bww4|GLLtR9sY-~D5wc}_?x#KBEy`4|?d8e1hZ@1v` zeuqE`rG3Na0QS$*(*d)W2o}f_T@l;kIeFchYZH7OF{qgm%?M6YbU>O*nPtv@X8_08 zfDotpt~Xbl_^V&%Iqva?gZLYUve$`!`ZDrI6eB?R^9oo)`uVnDVhpmu^vul37%_42 z3_jns7UkE6e)Z-4@$Z1cvu$|z7msuUzv~OzHGq7F-Pfe$4@q|5P7N=h@{Ha=GZnq= zpZTYJklM!M>q!5g_aE=;pDR!T`!I=AbCUa!+TZ>&aB%){0_pc)aySWGqp(Ggd=GKoAm1tXE&gWO*1 zB_g&u%rmLs5})tS`l-;SofhQ6w*pARiNDup(@6%)MnnNKE255>{mZ&YCR}iwfTe!8 zpyWF`kQiN;!OTugg_Bzat}q9W1HMEa3=+$Po%c|pwO@s>k3rzTlK27>%Nq+kY=(0I zxsA>coi{;m`Arz_WijHvtA#fBFasyf>yV<*BvU=ZTNMm~R~%Gl=;vhIE!{|l7Kqtg zeM+;ZZ6p#HLwu=$L&!H4MZ+t8jsD1aZOX!;WS;JQEr9e*8(u)e$wfLiR>v_PvnKMYHsjDcjVuQ&kx%x^rhryRvh6|COh{y1h0S*v zna1L0jA(1gLBqneD~laZ_SauM;_=e3y)B>ueNTm?&OBhq3Od_KanmQv2k|pFiCVut zMFza4gQ^oatwamRpn%X4M4lOr>nK`8pF4^lmBI}qy6oH6gYZ*f&;rVmhID?xnNvWS zvOJ2cr74L&iKrO>q4>asB?%?kW|@RgutsklL*W@ZXE+Hby?3qt@k!W!)zTe7;S|gJjmgnce6~mMw~NJ0dXJ;3>!sSo0X#UVZg?8?1(6~mf&PAmMgcF80n$+ z$`K-{@mC8Hl5fCoNJVVA23aj%LqQh;PX?m5T))@Ol`?Zs=it8G*A7~~Wv#wWxBKk9 zAaK-C7B*yYQ#WI$@u{sRJ-36&85+NH#9J}Zar*L&z08m}Fro+cgOiv*2s<-dAh{>- z7~qJq7ln0Z@IwYbFlBv;!`DpTyndx^VRu63jh7Ug|m6hR#Ac1cmCS>Ro0a&d(Um{dZm zTBJkcqh7uRX4;<45D#K~;nkdfv#$-O( Date: Fri, 29 Nov 2019 15:52:06 +0800 Subject: [PATCH 04/20] tools: refine language --- dev/how-to/maintain/backup-and-restore/br.md | 32 +++++++++---------- v3.1/how-to/maintain/backup-and-restore/br.md | 32 +++++++++---------- 2 files changed, 32 insertions(+), 32 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 7958bcea7fd0b..cf03158a2a23d 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -6,11 +6,11 @@ category: how-to # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with Mydumper/Loader, BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases of this tool. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases of this tool. ## Working principle -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding back and restoration operations. In an operation, each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. ![br-arch](/media/br-arch.png) @@ -26,8 +26,8 @@ In this command, * `backup` is the sub-command of `br`. * `full` is the sub-command of `backup`. -* `-s` or `--storage` specifies the path in which the backup files are stored. -* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the storage path in the local disk. +* `-s` or `--storage` specifies the directory in which the backup files are stored. +* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the directory in the local disk for storing backup files. * `--pd` is the PD service address. * `"${PDIP}:2379"` is the parameter of `--pd`. @@ -37,11 +37,11 @@ A `br` command consists of multiple layers of sub-command. Currently, BR has the * `br backup` is used to backup the data of the TiDB cluster. * `br restore` is used to restore the data of the TiDB cluster. -* `br version` is used to check the version of the BR tool. +* `br version` is used to check the version of BR. -Each of the above three sub-commands includes the following three available sub-commands: +Each of the above three sub-commands might include the following three sub-commands: -* `full` is used to backup or restore all data in the cluster. +* `full` is used to backup or restore all the cluster data. * `db` is used to restore the specified database of the cluster. * `table` is used to backup or restore a single table in the specified database of the cluster. @@ -62,7 +62,7 @@ To backup the cluster data, use the `br backup` command. You can add the `full` To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. -Use case: Backup all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. +Use case: Backup all the cluster data to the `/tmp/backup` directory of each TiKV node and write the `backupmeta` file to this directory. {{< copyable "shell-regular" >}} @@ -93,7 +93,7 @@ Full Backup <---------/................................................> 17.12%. To backup the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. -Use case: Backup the data of the `test.usertable` table to the `/tmp/backup` path in each TiKV node and write the `backupmeta` file to this path. +Use case: Backup the data of the `test.usertable` table to the `/tmp/backup` directory in each TiKV node and write the `backupmeta` file to this directory. {{< copyable "shell-regular" >}} @@ -110,7 +110,7 @@ br backup table \ The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. The other options have the same meanings as those in [Backup whole cluster](#backup-whole-cluster). -A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then, the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then, the BR also checks the backup data to ensure data security. ## Restore cluster data @@ -120,7 +120,7 @@ To restore the cluster data, use the `br restore` command. You can add the `full To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. -Usage: Restore all the backup data in the `/tmp/backup` path to the cluster. +Usage: Restore all the backup data in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -148,7 +148,7 @@ Full Restore <---------/...............................................> 17.12%. To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. -Usage: Restore a database in the backup data in the `/tmp/backup` path to the cluster. +Usage: Restore a database in the backup data in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -166,7 +166,7 @@ In the above command, `--db` specifies the name of the database to be restored. To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a database table in the backup data in the `/tmp/backup` path to the cluster. +Usage: Restore a database table in the backup data in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -183,7 +183,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Best practices -- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`. This makes it easier to collect and manage backup files. +- It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. This makes it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. - It is recommended that you perform the backup operation in a low-peak application time to minimize the impact on the application. @@ -191,8 +191,8 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. -- TiDB cannot perform the backup operation when executing DDL statements. -- The restoration can be performed only on new clusters. +- Do not perform the backup operation when executing DDL statements on TiDB. +- Perform the restoration only on new clusters. - If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `"10m0s"` by default, increase the value of this configuration. For example, set `tikv_gc_life_time` to `720h`: diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index ca72350fb5a53..4a3881b8954c8 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -6,11 +6,11 @@ category: how-to # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with Mydumper/Loader, BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases of this tool. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases of this tool. ## Working principle -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding back and restoration operations. In an operation, each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. ![br-arch](/media/br-arch.png) @@ -26,8 +26,8 @@ In this command, * `backup` is the sub-command of `br`. * `full` is the sub-command of `backup`. -* `-s` or `--storage` specifies the path in which the backup files are stored. -* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the storage path in the local disk. +* `-s` or `--storage` specifies the directory in which the backup files are stored. +* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the directory in the local disk for storing backup files. * `--pd` is the PD service address. * `"${PDIP}:2379"` is the parameter of `--pd`. @@ -37,11 +37,11 @@ A `br` command consists of multiple layers of sub-command. Currently, BR has the * `br backup` is used to backup the data of the TiDB cluster. * `br restore` is used to restore the data of the TiDB cluster. -* `br version` is used to check the version of the BR tool. +* `br version` is used to check the version of BR. -Each of the above three sub-commands includes the following three available sub-commands: +Each of the above three sub-commands might include the following three sub-commands: -* `full` is used to backup or restore all data in the cluster. +* `full` is used to backup or restore all the cluster data. * `db` is used to restore the specified database of the cluster. * `table` is used to backup or restore a single table in the specified database of the cluster. @@ -62,7 +62,7 @@ To backup the cluster data, use the `br backup` command. You can add the `full` To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. -Use case: Backup all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. +Use case: Backup all the cluster data to the `/tmp/backup` directory of each TiKV node and write the `backupmeta` file to this directory. {{< copyable "shell-regular" >}} @@ -93,7 +93,7 @@ Full Backup <---------/................................................> 17.12%. To backup the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. -Use case: Backup the data of the `test.usertable` table to the `/tmp/backup` path in each TiKV node and write the `backupmeta` file to this path. +Use case: Backup the data of the `test.usertable` table to the `/tmp/backup` directory in each TiKV node and write the `backupmeta` file to this directory. {{< copyable "shell-regular" >}} @@ -110,7 +110,7 @@ br backup table \ The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. The other options have the same meanings as those in [Backup whole cluster](#backup-whole-cluster). -A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then, the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then, the BR also checks the backup data to ensure data security. ## Restore cluster data @@ -120,7 +120,7 @@ To restore the cluster data, use the `br restore` command. You can add the `full To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. -Usage: Restore all the backup data in the `/tmp/backup` path to the cluster. +Usage: Restore all the backup data in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -148,7 +148,7 @@ Full Restore <---------/...............................................> 17.12%. To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. -Usage: Restore a database in the backup data in the `/tmp/backup` path to the cluster. +Usage: Restore a database in the backup data in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -166,7 +166,7 @@ In the above command, `--db` specifies the name of the database to be restored. To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a database table in the backup data in the `/tmp/backup` path to the cluster. +Usage: Restore a database table in the backup data in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -183,7 +183,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Best practices -- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`. This makes it easier to collect and manage backup files. +- It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. This makes it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. - It is recommended that you perform the backup operation in a low-peak application time to minimize the impact on the application. @@ -191,8 +191,8 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. -- TiDB cannot perform the backup operation when executing DDL statements. -- The restoration can be performed only on new clusters. +- Do not perform the backup operation when executing DDL statements on TiDB. +- Perform the restoration only on new clusters. - If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `"10m0s"` by default, increase the value of this configuration. For example, set `tikv_gc_life_time` to `720h`: From ce5e4f151ced9c0db187dfea7789a32c6b45d43d Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 29 Nov 2019 18:11:48 +0800 Subject: [PATCH 05/20] tools: refine language --- dev/how-to/maintain/backup-and-restore/br.md | 16 ++++++++-------- v3.1/how-to/maintain/backup-and-restore/br.md | 16 ++++++++-------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index cf03158a2a23d..ffcae94fc0806 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -75,7 +75,7 @@ br backup full \ --log-file backupfull.log ``` -In the above `br` command, the `--ratelimit` and `--concurrency` options set upper limits on the speed at which a backup operation is performed (MiB/s) and the number of concurrent executions for each TiKV node, and the BR log is written to the `backupfull.log` file. +In the above `br` command, the `--ratelimit` option specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. The `--concurrency` option sets an upper limit on the number of concurrent executions on each TiKV node. `--log-file` specifies that the BR log is written to the `restorefull.log` file. A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: @@ -108,9 +108,9 @@ br backup table \ --log-file backuptable.log ``` -The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. The other options have the same meanings as those in [Backup whole cluster](#backup-whole-cluster). +The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. For the meanings of other options, see [Backup all cluster data](#backup-all-cluster-data). -A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then, the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. ## Restore cluster data @@ -132,7 +132,7 @@ br restore full \ --log-file restorefull.log ``` -`--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation, and the BR log is written to the `restorefull.log` file. +In the above command, `--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation. `--log-file` specifies that the BR log is written to the `restorefull.log` file. A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then, the BR also checks the backup data to ensure data security. @@ -160,7 +160,7 @@ br restore db \ --log-file restorefull.log ``` -In the above command, `--db` specifies the name of the database to be restored. The other options have the same meaning as those in [Restore all backup data](#restore-all-backup-data). +In the above command, `--db` specifies the name of the database to be restored. For the meanings of other options, see [Restore all backup data](#restore-all-backup-data). ### Restore a table @@ -179,13 +179,13 @@ br restore table \ --log-file restorefull.log ``` -In the above command, `--table` specifies the name of the table to be restored. The other options have the same meaning as those in [Restore a database](#restore-a-database). +In the above command, `--table` specifies the name of the table to be restored. For the meanings of other options, see [Restore a database](#restore-a-database). ## Best practices - It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. This makes it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. -- It is recommended that you perform the backup operation in a low-peak application time to minimize the impact on the application. +- It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. ## Note @@ -193,7 +193,7 @@ In the above command, `--table` specifies the name of the table to be restored. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. - Do not perform the backup operation when executing DDL statements on TiDB. - Perform the restoration only on new clusters. -- If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `"10m0s"` by default, increase the value of this configuration. +- If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. For example, set `tikv_gc_life_time` to `720h`: diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 4a3881b8954c8..2fd746d26dc8a 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -75,7 +75,7 @@ br backup full \ --log-file backupfull.log ``` -In the above `br` command, the `--ratelimit` and `--concurrency` options set upper limits on the speed at which a backup operation is performed (MiB/s) and the number of concurrent executions for each TiKV node, and the BR log is written to the `backupfull.log` file. +In the above `br` command, the `--ratelimit` option specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. The `--concurrency` option sets an upper limit on the number of concurrent executions on each TiKV node. `--log-file` specifies that the BR log is written to the `restorefull.log` file. A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: @@ -108,9 +108,9 @@ br backup table \ --log-file backuptable.log ``` -The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. The other options have the same meanings as those in [Backup whole cluster](#backup-whole-cluster). +The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. For the meanings of other options, see [Backup all cluster data](#backup-all-cluster-data). -A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then, the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. ## Restore cluster data @@ -132,7 +132,7 @@ br restore full \ --log-file restorefull.log ``` -`--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation, and the BR log is written to the `restorefull.log` file. +In the above command, `--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation. `--log-file` specifies that the BR log is written to the `restorefull.log` file. A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then, the BR also checks the backup data to ensure data security. @@ -160,7 +160,7 @@ br restore db \ --log-file restorefull.log ``` -In the above command, `--db` specifies the name of the database to be restored. The other options have the same meaning as those in [Restore all backup data](#restore-all-backup-data). +In the above command, `--db` specifies the name of the database to be restored. For the meanings of other options, see [Restore all backup data](#restore-all-backup-data). ### Restore a table @@ -179,13 +179,13 @@ br restore table \ --log-file restorefull.log ``` -In the above command, `--table` specifies the name of the table to be restored. The other options have the same meaning as those in [Restore a database](#restore-a-database). +In the above command, `--table` specifies the name of the table to be restored. For the meanings of other options, see [Restore a database](#restore-a-database). ## Best practices - It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. This makes it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. -- It is recommended that you perform the backup operation in a low-peak application time to minimize the impact on the application. +- It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. ## Note @@ -193,7 +193,7 @@ In the above command, `--table` specifies the name of the table to be restored. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. - Do not perform the backup operation when executing DDL statements on TiDB. - Perform the restoration only on new clusters. -- If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `"10m0s"` by default, increase the value of this configuration. +- If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. For example, set `tikv_gc_life_time` to `720h`: From ba15be8a38e8414b4c4ca9f735d0385e8fd0b40c Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Mon, 2 Dec 2019 19:48:55 +0800 Subject: [PATCH 06/20] maintain: update content --- dev/how-to/maintain/backup-and-restore/br.md | 91 ++++++++++++++++++- v3.1/how-to/maintain/backup-and-restore/br.md | 91 ++++++++++++++++++- 2 files changed, 180 insertions(+), 2 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index ffcae94fc0806..87646940b9bed 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -192,7 +192,8 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. - Do not perform the backup operation when executing DDL statements on TiDB. -- Perform the restoration only on new clusters. +- Currently you cannot perform the backup and restoration on the partition table. +- Currently you can perform the restoration only on new clusters. - If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. For example, set `tikv_gc_life_time` to `720h`: @@ -225,3 +226,91 @@ In the above command, `--table` specifies the name of the table to be restored. ./pd-ctl -u ${PDIP}:2379 scheduler add balance-leader-scheduler ./pd-ctl -u ${PDIP}:2379 scheduler add balance-region-scheduler ``` + +## Examples + +This section shows how to backup and restore the data of an existing cluster. + +### Data source and machine configuration + +Suppose that the backup and restoration operations are performed on 10 tables in the TiKV cluster, each table with 5 million rows of data. The total data size is 35 GB. + +```sql +MySQL [sbtest]> show tables; ++------------------+ +| Tables_in_sbtest | ++------------------+ +| sbtest1 | +| sbtest10 | +| sbtest2 | +| sbtest3 | +| sbtest4 | +| sbtest5 | +| sbtest6 | +| sbtest7 | +| sbtest8 | +| sbtest9 | ++------------------+ + +MySQL [sbtest]> select count(*) from sbtest1; ++----------+ +| count(*) | ++----------+ +| 5000000 | ++----------+ +1 row in set (1.04 sec) +``` + +The table structure is as follows: + +```sql +CREATE TABLE `sbtest1` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `k` int(11) NOT NULL DEFAULT '0', + `c` char(120) NOT NULL DEFAULT '', + `pad` char(60) NOT NULL DEFAULT '', + PRIMARY KEY (`id`), + KEY `k_1` (`k`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin AUTO_INCREMENT=5138499 +``` + +Suppose that 4 TiKV nodes is used, each with the following configuration: + +16 Core，32GB MEM，SSD, 3 replicas + +### Backup + +Before the backup operation, make sure the following things are done: + +- `tikv_gc_life_time` is set to a larger value so that the backup operation will not be interrupted by data loss. +- No DDL statement is executed on TiDB. + +Then execute the following command to backup all the cluster data: + +{{< copyable "shell-regular" >}} + +``` +bin/br backup full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file backup.log +``` + +``` +[INFO] [client.go:288] ["Backup Ranges"] [take=2m25.801322134s] +[INFO] [schema.go:114] ["backup checksum finished"] [take=4.842154366s] +``` + +### Restoration + +Before the restoration, make sure that the TiKV cluster to be restored is a new cluster. + +Then execute the following command to restore all the cluster data: + +{{< copyable "shell-regular" >}} + +``` +bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restore.log +``` + +``` +[INFO] [client.go:345] [RestoreAll] [take=2m8.907369337s] +[INFO] [client.go:435] ["Restore Checksum"] [take=6.385818026s] +``` \ No newline at end of file diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 2fd746d26dc8a..a111fa28abead 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -192,7 +192,8 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. - Do not perform the backup operation when executing DDL statements on TiDB. -- Perform the restoration only on new clusters. +- Currently you cannot perform the backup and restoration on the partition table. +- Currently you can perform the restoration only on new clusters. - If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. For example, set `tikv_gc_life_time` to `720h`: @@ -225,3 +226,91 @@ In the above command, `--table` specifies the name of the table to be restored. ./pd-ctl -u ${PDIP}:2379 scheduler add balance-leader-scheduler ./pd-ctl -u ${PDIP}:2379 scheduler add balance-region-scheduler ``` + +## Examples + +This section shows how to backup and restore the data of an existing cluster. + +### Data source and machine configuration + +Suppose that the backup and restoration operations are performed on 10 tables in the TiKV cluster, each table with 5 million rows of data. The total data size is 35 GB. + +```sql +MySQL [sbtest]> show tables; ++------------------+ +| Tables_in_sbtest | ++------------------+ +| sbtest1 | +| sbtest10 | +| sbtest2 | +| sbtest3 | +| sbtest4 | +| sbtest5 | +| sbtest6 | +| sbtest7 | +| sbtest8 | +| sbtest9 | ++------------------+ + +MySQL [sbtest]> select count(*) from sbtest1; ++----------+ +| count(*) | ++----------+ +| 5000000 | ++----------+ +1 row in set (1.04 sec) +``` + +The table structure is as follows: + +```sql +CREATE TABLE `sbtest1` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `k` int(11) NOT NULL DEFAULT '0', + `c` char(120) NOT NULL DEFAULT '', + `pad` char(60) NOT NULL DEFAULT '', + PRIMARY KEY (`id`), + KEY `k_1` (`k`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin AUTO_INCREMENT=5138499 +``` + +Suppose that 4 TiKV nodes is used, each with the following configuration: + +16 Core，32GB MEM，SSD, 3 replicas + +### Backup + +Before the backup operation, make sure the following things are done: + +- `tikv_gc_life_time` is set to a larger value so that the backup operation will not be interrupted by data loss. +- No DDL statement is executed on TiDB. + +Then execute the following command to backup all the cluster data: + +{{< copyable "shell-regular" >}} + +``` +bin/br backup full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file backup.log +``` + +``` +[INFO] [client.go:288] ["Backup Ranges"] [take=2m25.801322134s] +[INFO] [schema.go:114] ["backup checksum finished"] [take=4.842154366s] +``` + +### Restoration + +Before the restoration, make sure that the TiKV cluster to be restored is a new cluster. + +Then execute the following command to restore all the cluster data: + +{{< copyable "shell-regular" >}} + +``` +bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restore.log +``` + +``` +[INFO] [client.go:345] [RestoreAll] [take=2m8.907369337s] +[INFO] [client.go:435] ["Restore Checksum"] [take=6.385818026s] +``` \ No newline at end of file From 08055556a1f4d8558efc62ebb9bd6ba4affc8106 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Mon, 2 Dec 2019 20:18:34 +0800 Subject: [PATCH 07/20] maintain: refine language --- dev/how-to/maintain/backup-and-restore/br.md | 16 ++++++++-------- v3.1/how-to/maintain/backup-and-restore/br.md | 14 +++++++------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 87646940b9bed..cd79add7de63c 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -6,7 +6,7 @@ category: how-to # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases of this tool. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases and best practices of this tool. ## Working principle @@ -33,13 +33,13 @@ In this command, ### Sub-commands -A `br` command consists of multiple layers of sub-command. Currently, BR has the following three sub-commands: +A `br` command consists of multiple layers of sub-commands. Currently, BR has the following three sub-commands: * `br backup` is used to backup the data of the TiDB cluster. * `br restore` is used to restore the data of the TiDB cluster. * `br version` is used to check the version of BR. -Each of the above three sub-commands might include the following three sub-commands: +Each of the above three sub-commands might include the following three sub-commands to specify the scope of an operation: * `full` is used to backup or restore all the cluster data. * `db` is used to restore the specified database of the cluster. @@ -183,15 +183,15 @@ In the above command, `--table` specifies the name of the table to be restored. ## Best practices -- It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. This makes it easier to collect and manage backup files. +- It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. Then you will find it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. - It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. ## Note -- BR only supports TiDB v3.1 or later versions. +- BR only supports TiDB dev or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. -- Do not perform the backup operation when executing DDL statements on TiDB. +- Do not perform the backup operation when performing DDL operations on TiDB. - Currently you cannot perform the backup and restoration on the partition table. - Currently you can perform the restoration only on new clusters. - If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. @@ -276,7 +276,7 @@ CREATE TABLE `sbtest1` ( Suppose that 4 TiKV nodes is used, each with the following configuration: -16 Core，32GB MEM，SSD, 3 replicas +16 Core, 32GB MEM, SSD, 3 replicas ### Backup @@ -313,4 +313,4 @@ bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restor ``` [INFO] [client.go:345] [RestoreAll] [take=2m8.907369337s] [INFO] [client.go:435] ["Restore Checksum"] [take=6.385818026s] -``` \ No newline at end of file +``` diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index a111fa28abead..a36d553fbc231 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -6,7 +6,7 @@ category: how-to # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases of this tool. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases and best practices of this tool. ## Working principle @@ -33,13 +33,13 @@ In this command, ### Sub-commands -A `br` command consists of multiple layers of sub-command. Currently, BR has the following three sub-commands: +A `br` command consists of multiple layers of sub-commands. Currently, BR has the following three sub-commands: * `br backup` is used to backup the data of the TiDB cluster. * `br restore` is used to restore the data of the TiDB cluster. * `br version` is used to check the version of BR. -Each of the above three sub-commands might include the following three sub-commands: +Each of the above three sub-commands might include the following three sub-commands to specify the scope of an operation: * `full` is used to backup or restore all the cluster data. * `db` is used to restore the specified database of the cluster. @@ -183,7 +183,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Best practices -- It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. This makes it easier to collect and manage backup files. +- It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. Then you will find it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. - It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. @@ -191,7 +191,7 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. -- Do not perform the backup operation when executing DDL statements on TiDB. +- Do not perform the backup operation when performing DDL operations on TiDB. - Currently you cannot perform the backup and restoration on the partition table. - Currently you can perform the restoration only on new clusters. - If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. @@ -276,7 +276,7 @@ CREATE TABLE `sbtest1` ( Suppose that 4 TiKV nodes is used, each with the following configuration: -16 Core，32GB MEM，SSD, 3 replicas +16 Core, 32GB MEM, SSD, 3 replicas ### Backup @@ -313,4 +313,4 @@ bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restor ``` [INFO] [client.go:345] [RestoreAll] [take=2m8.907369337s] [INFO] [client.go:435] ["Restore Checksum"] [take=6.385818026s] -``` \ No newline at end of file +``` From 2c4dc01a6df40f7aea5356ae22511ec7da652e81 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Mon, 2 Dec 2019 20:34:35 +0800 Subject: [PATCH 08/20] maintain: fix a typo --- dev/how-to/maintain/backup-and-restore/br.md | 6 +++--- v3.1/how-to/maintain/backup-and-restore/br.md | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index cd79add7de63c..885511c50e875 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -148,7 +148,7 @@ Full Restore <---------/...............................................> 17.12%. To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. -Usage: Restore a database in the backup data in the `/tmp/backup` directory to the cluster. +Usage: Restore a database backed up in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -166,7 +166,7 @@ In the above command, `--db` specifies the name of the database to be restored. To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a database table in the backup data in the `/tmp/backup` directory to the cluster. +Usage: Restore a table backed up in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -189,7 +189,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Note -- BR only supports TiDB dev or later versions. +- BR only supports TiDB v3.1 or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. - Do not perform the backup operation when performing DDL operations on TiDB. - Currently you cannot perform the backup and restoration on the partition table. diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index a36d553fbc231..6c5639f7f3412 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -148,7 +148,7 @@ Full Restore <---------/...............................................> 17.12%. To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. -Usage: Restore a database in the backup data in the `/tmp/backup` directory to the cluster. +Usage: Restore a database backed up in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -166,7 +166,7 @@ In the above command, `--db` specifies the name of the database to be restored. To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a database table in the backup data in the `/tmp/backup` directory to the cluster. +Usage: Restore a table backed up data in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} From ea9bce0b72734de9907f7e0f95a7bbf2cf55f0e2 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Tue, 3 Dec 2019 20:12:01 +0800 Subject: [PATCH 09/20] maintain: add working principles --- dev/how-to/maintain/backup-and-restore/br.md | 65 +++++++++++++++--- v3.1/how-to/maintain/backup-and-restore/br.md | 67 ++++++++++++++++--- 2 files changed, 115 insertions(+), 17 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 885511c50e875..3e49c17f11503 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -8,12 +8,6 @@ category: how-to Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases and best practices of this tool. -## Working principle - -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. - -![br-arch](/media/br-arch.png) - ## Command-line description A `br` command consists of sub-commands, options and parameters. The sub-command is the characters without `-` or `--`. The option is the characters that start with `-` or `--`. The parameter is the characters that immediately follow and pass to the sub-command or the option. @@ -28,7 +22,7 @@ In this command, * `full` is the sub-command of `backup`. * `-s` or `--storage` specifies the directory in which the backup files are stored. * `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the directory in the local disk for storing backup files. -* `--pd` is the PD service address. +* `--pd` is the Placement Driver (PD) service address. * `"${PDIP}:2379"` is the parameter of `--pd`. ### Sub-commands @@ -227,9 +221,64 @@ In the above command, `--table` specifies the name of the table to be restored. ./pd-ctl -u ${PDIP}:2379 scheduler add balance-region-scheduler ``` +## Working principles + +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. + +### Backup + +When BR performs a backup operation, it first obtains the following information from the PD: + +- The current TS (timestamp) as the time of the backup snapshot +- The TiKV node information of the current cluster + +According to these information, BR starts a TiDB instance internally to obtain the database or table information corresponding to the TS, and filters out the system databases (`information_schema`, `performance_schema`, `mysql`) at the same time. + +According to the backup sub-command, two types of backup logic are available: + +- Full backup: BR traverses all the tables and constructs the KV ranges to be backed up according to each table. +- Single table backup: BR constructs the KV range to be backed up according a single table. + +Finally, BR collects the KV range to be backed up and sends the complete backup requests to the TiKV nodes in the cluster. + +The structure of the request: + +``` +backup.BackupRequest{ + ClusterId: clusterID, // The cluster ID + StartKey: startKey, // The starting key of the backup (backed up) + EndKey: endKey, // The ending key of the backup (not backed up) + StartVersion: backupTS, // The backup snapshot time + ... + Path: path, // The path in which backup files are stored + RateLimit: rateLimit, // Backup speed (MB/s) + Concurrency: concurrency, // The number of threads for the backup operation (4 by default) +} +``` + +After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backs up some or all of the data within the range, and generates the corresponding SST file in the backup path (named in the format of `storeID_regionID_regionEpoch_tableID`). + +After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. + +If checksum is enabled when you execute the backup command, BR calculates the checksum of each backed up table for data check. + +### Restoration + +When BR performs the restoration, it first parse the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. BR aggregates the parsed SST file according to the tables and `GroupBy`. + +BR pre-splits Regions according to the key range of the SST file so that every Region corresponds to an SST file. + +BR traverses each table to be restored and the SST file corresponding to these tables. BR finds the Region corresponding to the SST file and sends a request for downloading the file to the corresponding TiKV node. Then BR sends a request for loading the file after the file is successfully downloaded. + +After TiKV receives the request to load the SST file, it uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. + +After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. + +![br-arch](/media/br-arch.png) + ## Examples -This section shows how to backup and restore the data of an existing cluster. +This section shows how to backup and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. ### Data source and machine configuration diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 6c5639f7f3412..6976b96bd1ca7 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -8,12 +8,6 @@ category: how-to Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases and best practices of this tool. -## Working principle - -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. - -![br-arch](/media/br-arch.png) - ## Command-line description A `br` command consists of sub-commands, options and parameters. The sub-command is the characters without `-` or `--`. The option is the characters that start with `-` or `--`. The parameter is the characters that immediately follow and pass to the sub-command or the option. @@ -28,7 +22,7 @@ In this command, * `full` is the sub-command of `backup`. * `-s` or `--storage` specifies the directory in which the backup files are stored. * `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the directory in the local disk for storing backup files. -* `--pd` is the PD service address. +* `--pd` is the Placement Driver (PD) service address. * `"${PDIP}:2379"` is the parameter of `--pd`. ### Sub-commands @@ -166,7 +160,7 @@ In the above command, `--db` specifies the name of the database to be restored. To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a table backed up data in the `/tmp/backup` directory to the cluster. +Usage: Restore a table backed up in the `/tmp/backup` directory to the cluster. {{< copyable "shell-regular" >}} @@ -227,9 +221,64 @@ In the above command, `--table` specifies the name of the table to be restored. ./pd-ctl -u ${PDIP}:2379 scheduler add balance-region-scheduler ``` +## Working principles + +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. + +### Backup + +When BR performs a backup operation, it first obtains the following information from the PD: + +- The current TS (timestamp) as the time of the backup snapshot +- The TiKV node information of the current cluster + +According to these information, BR starts a TiDB instance internally to obtain the database or table information corresponding to the TS, and filters out the system databases (`information_schema`, `performance_schema`, `mysql`) at the same time. + +According to the backup sub-command, two types of backup logic are available: + +- Full backup: BR traverses all the tables and constructs the KV ranges to be backed up according to each table. +- Single table backup: BR constructs the KV range to be backed up according a single table. + +Finally, BR collects the KV range to be backed up and sends the complete backup requests to the TiKV nodes in the cluster. + +The structure of the request: + +``` +backup.BackupRequest{ + ClusterId: clusterID, // The cluster ID + StartKey: startKey, // The starting key of the backup (backed up) + EndKey: endKey, // The ending key of the backup (not backed up) + StartVersion: backupTS, // The backup snapshot time + ... + Path: path, // The path in which backup files are stored + RateLimit: rateLimit, // Backup speed (MB/s) + Concurrency: concurrency, // The number of threads for the backup operation (4 by default) +} +``` + +After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backs up some or all of the data within the range, and generates the corresponding SST file in the backup path (named in the format of `storeID_regionID_regionEpoch_tableID`). + +After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. + +If checksum is enabled when you execute the backup command, BR calculates the checksum of each backed up table for data check. + +### Restoration + +When BR performs the restoration, it first parse the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. BR aggregates the parsed SST file according to the tables and `GroupBy`. + +BR pre-splits Regions according to the key range of the SST file so that every Region corresponds to an SST file. + +BR traverses each table to be restored and the SST file corresponding to these tables. BR finds the Region corresponding to the SST file and sends a request for downloading the file to the corresponding TiKV node. Then BR sends a request for loading the file after the file is successfully downloaded. + +After TiKV receives the request to load the SST file, it uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. + +After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. + +![br-arch](/media/br-arch.png) + ## Examples -This section shows how to backup and restore the data of an existing cluster. +This section shows how to backup and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. ### Data source and machine configuration From 3a0a9e0668aaf2464404c550de19bca863906a31 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Tue, 3 Dec 2019 20:34:07 +0800 Subject: [PATCH 10/20] maintain: refine introduction --- dev/how-to/maintain/backup-and-restore/br.md | 2 +- v3.1/how-to/maintain/backup-and-restore/br.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 3e49c17f11503..d9bf1d9bdcb72 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -6,7 +6,7 @@ category: how-to # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases and best practices of this tool. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the command line, offers detailed use cases and best practices, and introduces the working principle of BR. ## Command-line description diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 6976b96bd1ca7..ec073b9f66e54 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -6,7 +6,7 @@ category: how-to # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [Mydumper/Loader](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document introduces the working principle of BR, describes the command line, and offers detailed use cases and best practices of this tool. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the command line, offers detailed use cases and best practices, and introduces the working principle of BR. ## Command-line description From e93a47f87d70d14e01c6fe7ac461615cf2185414 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Wed, 4 Dec 2019 11:09:29 +0800 Subject: [PATCH 11/20] maintain: refine format and layout --- dev/how-to/maintain/backup-and-restore/br.md | 28 ++++++++++++------- v3.1/how-to/maintain/backup-and-restore/br.md | 28 ++++++++++++------- 2 files changed, 36 insertions(+), 20 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index d9bf1d9bdcb72..ab67877e09c1b 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -185,8 +185,8 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. -- Do not perform the backup operation when performing DDL operations on TiDB. -- Currently you cannot perform the backup and restoration on the partition table. +- TiDB cannot perform the backup operation when executing DDL operations. +- Currently TiDB does not support the backup and restoration of the partition table. - Currently you can perform the restoration only on new clusters. - If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. @@ -236,10 +236,10 @@ According to these information, BR starts a TiDB instance internally to obtain t According to the backup sub-command, two types of backup logic are available: -- Full backup: BR traverses all the tables and constructs the KV ranges to be backed up according to each table. +- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to every table. - Single table backup: BR constructs the KV range to be backed up according a single table. -Finally, BR collects the KV range to be backed up and sends the complete backup requests to the TiKV nodes in the cluster. +Finally, BR collects the KV range to be backed up and sends the complete backup request to the TiKV node of the cluster. The structure of the request: @@ -256,7 +256,7 @@ backup.BackupRequest{ } ``` -After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backs up some or all of the data within the range, and generates the corresponding SST file in the backup path (named in the format of `storeID_regionID_regionEpoch_tableID`). +After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backups some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. @@ -264,13 +264,19 @@ If checksum is enabled when you execute the backup command, BR calculates the ch ### Restoration -When BR performs the restoration, it first parse the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. BR aggregates the parsed SST file according to the tables and `GroupBy`. +When BR performs the restoration, BR performs the following tasks in order: -BR pre-splits Regions according to the key range of the SST file so that every Region corresponds to an SST file. +1. It parses the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. -BR traverses each table to be restored and the SST file corresponding to these tables. BR finds the Region corresponding to the SST file and sends a request for downloading the file to the corresponding TiKV node. Then BR sends a request for loading the file after the file is successfully downloaded. +2. It aggregates the parsed SST file according to the tables and `GroupBy`. -After TiKV receives the request to load the SST file, it uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. +3. It pre-splits Regions according to the key range of the SST file so that every Region corresponds to an SST file. + +4. It traverses every table to be restored and the SST file corresponding to these tables. + +5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. + +After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. @@ -325,7 +331,9 @@ CREATE TABLE `sbtest1` ( Suppose that 4 TiKV nodes is used, each with the following configuration: -16 Core, 32GB MEM, SSD, 3 replicas +| CPU | Memory | Disk | Number of replicas | +| :-------- | :------ | :---- | :------------------ | +| 16 cores | 32 GB | SSD | 3 | ### Backup diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index ec073b9f66e54..31ea39ec2514a 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -185,8 +185,8 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. -- Do not perform the backup operation when performing DDL operations on TiDB. -- Currently you cannot perform the backup and restoration on the partition table. +- TiDB cannot perform the backup operation when executing DDL operations. +- Currently TiDB does not support the backup and restoration of the partition table. - Currently you can perform the restoration only on new clusters. - If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. @@ -236,10 +236,10 @@ According to these information, BR starts a TiDB instance internally to obtain t According to the backup sub-command, two types of backup logic are available: -- Full backup: BR traverses all the tables and constructs the KV ranges to be backed up according to each table. +- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to every table. - Single table backup: BR constructs the KV range to be backed up according a single table. -Finally, BR collects the KV range to be backed up and sends the complete backup requests to the TiKV nodes in the cluster. +Finally, BR collects the KV range to be backed up and sends the complete backup request to the TiKV node of the cluster. The structure of the request: @@ -256,7 +256,7 @@ backup.BackupRequest{ } ``` -After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backs up some or all of the data within the range, and generates the corresponding SST file in the backup path (named in the format of `storeID_regionID_regionEpoch_tableID`). +After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backups some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. @@ -264,13 +264,19 @@ If checksum is enabled when you execute the backup command, BR calculates the ch ### Restoration -When BR performs the restoration, it first parse the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. BR aggregates the parsed SST file according to the tables and `GroupBy`. +When BR performs the restoration, BR performs the following tasks in order: -BR pre-splits Regions according to the key range of the SST file so that every Region corresponds to an SST file. +1. It parses the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. -BR traverses each table to be restored and the SST file corresponding to these tables. BR finds the Region corresponding to the SST file and sends a request for downloading the file to the corresponding TiKV node. Then BR sends a request for loading the file after the file is successfully downloaded. +2. It aggregates the parsed SST file according to the tables and `GroupBy`. -After TiKV receives the request to load the SST file, it uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. +3. It pre-splits Regions according to the key range of the SST file so that every Region corresponds to an SST file. + +4. It traverses every table to be restored and the SST file corresponding to these tables. + +5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. + +After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. @@ -325,7 +331,9 @@ CREATE TABLE `sbtest1` ( Suppose that 4 TiKV nodes is used, each with the following configuration: -16 Core, 32GB MEM, SSD, 3 replicas +| CPU | Memory | Disk | Number of replicas | +| :-------- | :------ | :---- | :------------------ | +| 16 cores | 32 GB | SSD | 3 | ### Backup From a27a1c70454a2620847325099e1f91fbd5519fa4 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Wed, 4 Dec 2019 13:49:05 +0800 Subject: [PATCH 12/20] maintain: address comments from #2088 --- dev/how-to/maintain/backup-and-restore/br.md | 163 +++++++++--------- v3.1/how-to/maintain/backup-and-restore/br.md | 163 +++++++++--------- 2 files changed, 166 insertions(+), 160 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index ab67877e09c1b..9776ed5b141e2 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -6,7 +6,7 @@ category: how-to # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the command line, offers detailed use cases and best practices, and introduces the working principle of BR. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the command line, offers detailed use cases, best practices, usage restrictions, and introduces the working principle of BR. ## Command-line description @@ -52,6 +52,17 @@ Each of the above three sub-commands might include the following three sub-comma To backup the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table. +If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. + +For example, set `tikv_gc_life_time` to `720h`: + +{{< copyable "sql" >}} + +``` +mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ + "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; +``` + ### Backup all cluster data To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. @@ -110,6 +121,10 @@ A progress bar is displayed in the terminal during the backup operation. When th To restore the cluster data, use the `br restore` command. You can add the `full`, `db` or `table` sub-command to specify the scope of your restoration: the whole cluster, a database or a single table. +> **Note:** +> +> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. + ### Restore all backup data To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. @@ -180,25 +195,6 @@ In the above command, `--table` specifies the name of the table to be restored. - It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. Then you will find it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. - It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. - -## Note - -- BR only supports TiDB v3.1 or later versions. -- If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. -- TiDB cannot perform the backup operation when executing DDL operations. -- Currently TiDB does not support the backup and restoration of the partition table. -- Currently you can perform the restoration only on new clusters. -- If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. - - For example, set `tikv_gc_life_time` to `720h`: - - {{< copyable "sql" >}} - - ``` - mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ - "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; - ``` - - To speed up the restoration, use pd-ctl to remove the schedulers related to scheduling before the restoration and add back these removed schedulers after the restoration. Remove schedulers: @@ -221,72 +217,18 @@ In the above command, `--table` specifies the name of the table to be restored. ./pd-ctl -u ${PDIP}:2379 scheduler add balance-region-scheduler ``` -## Working principles - -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. - -### Backup - -When BR performs a backup operation, it first obtains the following information from the PD: - -- The current TS (timestamp) as the time of the backup snapshot -- The TiKV node information of the current cluster - -According to these information, BR starts a TiDB instance internally to obtain the database or table information corresponding to the TS, and filters out the system databases (`information_schema`, `performance_schema`, `mysql`) at the same time. - -According to the backup sub-command, two types of backup logic are available: - -- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to every table. -- Single table backup: BR constructs the KV range to be backed up according a single table. +## Usage restrictions -Finally, BR collects the KV range to be backed up and sends the complete backup request to the TiKV node of the cluster. - -The structure of the request: - -``` -backup.BackupRequest{ - ClusterId: clusterID, // The cluster ID - StartKey: startKey, // The starting key of the backup (backed up) - EndKey: endKey, // The ending key of the backup (not backed up) - StartVersion: backupTS, // The backup snapshot time - ... - Path: path, // The path in which backup files are stored - RateLimit: rateLimit, // Backup speed (MB/s) - Concurrency: concurrency, // The number of threads for the backup operation (4 by default) -} -``` - -After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backups some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. - -After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. - -If checksum is enabled when you execute the backup command, BR calculates the checksum of each backed up table for data check. - -### Restoration - -When BR performs the restoration, BR performs the following tasks in order: - -1. It parses the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. - -2. It aggregates the parsed SST file according to the tables and `GroupBy`. - -3. It pre-splits Regions according to the key range of the SST file so that every Region corresponds to an SST file. - -4. It traverses every table to be restored and the SST file corresponding to these tables. - -5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. - -After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. - -After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. - -![br-arch](/media/br-arch.png) +- BR only supports TiDB v3.1 or later versions. +- TiDB cannot perform the backup operation when executing DDL operations. +- Currently TiDB does not support the backup and restoration of the partition table. +- Currently you can perform the restoration only on new clusters. ## Examples This section shows how to backup and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. -### Data source and machine configuration +### Data size and machine configuration Suppose that the backup and restoration operations are performed on 10 tables in the TiKV cluster, each table with 5 million rows of data. The total data size is 35 GB. @@ -371,3 +313,64 @@ bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restor [INFO] [client.go:345] [RestoreAll] [take=2m8.907369337s] [INFO] [client.go:435] ["Restore Checksum"] [take=6.385818026s] ``` + +## Working principles + +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. + +### Backup + +When BR performs a backup operation, it first obtains the following information from the PD: + +- The current TS (timestamp) as the time of the backup snapshot +- The TiKV node information of the current cluster + +According to these information, BR starts a TiDB instance internally to obtain the database or table information corresponding to the TS, and filters out the system databases (`information_schema`, `performance_schema`, `mysql`) at the same time. + +According to the backup sub-command, two types of backup logic are available: + +- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to every table. +- Single table backup: BR constructs the KV range to be backed up according a single table. + +Finally, BR collects the KV range to be backed up and sends the complete backup request to the TiKV node of the cluster. + +The structure of the request: + +``` +backup.BackupRequest{ + ClusterId: clusterID, // The cluster ID + StartKey: startKey, // The starting key of the backup (backed up) + EndKey: endKey, // The ending key of the backup (not backed up) + StartVersion: backupTS, // The backup snapshot time + ... + Path: path, // The path in which backup files are stored + RateLimit: rateLimit, // Backup speed (MB/s) + Concurrency: concurrency, // The number of threads for the backup operation (4 by default) +} +``` + +After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backups some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. + +After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. + +If checksum is enabled when you execute the backup command, BR calculates the checksum of each backed up table for data check. + +### Restoration + +When BR performs the restoration, BR performs the following tasks in order: + +1. It parses the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. + +2. It aggregates the parsed SST file according to the tables and `GroupBy`. + +3. It pre-splits Regions according to the key range of the SST file so that every Region corresponds to at least one SST file. + +4. It traverses every table to be restored and the SST file corresponding to these tables. + +5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. + +After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. + +After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. + +![br-arch](/media/br-arch.png) diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 31ea39ec2514a..3dbbdf573c16d 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -6,7 +6,7 @@ category: how-to # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the command line, offers detailed use cases and best practices, and introduces the working principle of BR. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the command line, offers detailed use cases, best practices, usage restrictions, and introduces the working principle of BR. ## Command-line description @@ -52,6 +52,17 @@ Each of the above three sub-commands might include the following three sub-comma To backup the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table. +If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. + +For example, set `tikv_gc_life_time` to `720h`: + +{{< copyable "sql" >}} + +``` +mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ + "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; +``` + ### Backup all cluster data To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. @@ -110,6 +121,10 @@ A progress bar is displayed in the terminal during the backup operation. When th To restore the cluster data, use the `br restore` command. You can add the `full`, `db` or `table` sub-command to specify the scope of your restoration: the whole cluster, a database or a single table. +> **Note:** +> +> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. + ### Restore all backup data To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. @@ -180,25 +195,6 @@ In the above command, `--table` specifies the name of the table to be restored. - It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. Then you will find it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. - It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. - -## Note - -- BR only supports TiDB v3.1 or later versions. -- If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. -- TiDB cannot perform the backup operation when executing DDL operations. -- Currently TiDB does not support the backup and restoration of the partition table. -- Currently you can perform the restoration only on new clusters. -- If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. - - For example, set `tikv_gc_life_time` to `720h`: - - {{< copyable "sql" >}} - - ``` - mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ - "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; - ``` - - To speed up the restoration, use pd-ctl to remove the schedulers related to scheduling before the restoration and add back these removed schedulers after the restoration. Remove schedulers: @@ -221,72 +217,18 @@ In the above command, `--table` specifies the name of the table to be restored. ./pd-ctl -u ${PDIP}:2379 scheduler add balance-region-scheduler ``` -## Working principles - -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. - -### Backup - -When BR performs a backup operation, it first obtains the following information from the PD: - -- The current TS (timestamp) as the time of the backup snapshot -- The TiKV node information of the current cluster - -According to these information, BR starts a TiDB instance internally to obtain the database or table information corresponding to the TS, and filters out the system databases (`information_schema`, `performance_schema`, `mysql`) at the same time. - -According to the backup sub-command, two types of backup logic are available: - -- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to every table. -- Single table backup: BR constructs the KV range to be backed up according a single table. +## Usage restrictions -Finally, BR collects the KV range to be backed up and sends the complete backup request to the TiKV node of the cluster. - -The structure of the request: - -``` -backup.BackupRequest{ - ClusterId: clusterID, // The cluster ID - StartKey: startKey, // The starting key of the backup (backed up) - EndKey: endKey, // The ending key of the backup (not backed up) - StartVersion: backupTS, // The backup snapshot time - ... - Path: path, // The path in which backup files are stored - RateLimit: rateLimit, // Backup speed (MB/s) - Concurrency: concurrency, // The number of threads for the backup operation (4 by default) -} -``` - -After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backups some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. - -After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. - -If checksum is enabled when you execute the backup command, BR calculates the checksum of each backed up table for data check. - -### Restoration - -When BR performs the restoration, BR performs the following tasks in order: - -1. It parses the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. - -2. It aggregates the parsed SST file according to the tables and `GroupBy`. - -3. It pre-splits Regions according to the key range of the SST file so that every Region corresponds to an SST file. - -4. It traverses every table to be restored and the SST file corresponding to these tables. - -5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. - -After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. - -After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. - -![br-arch](/media/br-arch.png) +- BR only supports TiDB v3.1 or later versions. +- TiDB cannot perform the backup operation when executing DDL operations. +- Currently TiDB does not support the backup and restoration of the partition table. +- Currently you can perform the restoration only on new clusters. ## Examples This section shows how to backup and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. -### Data source and machine configuration +### Data size and machine configuration Suppose that the backup and restoration operations are performed on 10 tables in the TiKV cluster, each table with 5 million rows of data. The total data size is 35 GB. @@ -371,3 +313,64 @@ bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restor [INFO] [client.go:345] [RestoreAll] [take=2m8.907369337s] [INFO] [client.go:435] ["Restore Checksum"] [take=6.385818026s] ``` + +## Working principles + +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. + +### Backup + +When BR performs a backup operation, it first obtains the following information from the PD: + +- The current TS (timestamp) as the time of the backup snapshot +- The TiKV node information of the current cluster + +According to these information, BR starts a TiDB instance internally to obtain the database or table information corresponding to the TS, and filters out the system databases (`information_schema`, `performance_schema`, `mysql`) at the same time. + +According to the backup sub-command, two types of backup logic are available: + +- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to every table. +- Single table backup: BR constructs the KV range to be backed up according a single table. + +Finally, BR collects the KV range to be backed up and sends the complete backup request to the TiKV node of the cluster. + +The structure of the request: + +``` +backup.BackupRequest{ + ClusterId: clusterID, // The cluster ID + StartKey: startKey, // The starting key of the backup (backed up) + EndKey: endKey, // The ending key of the backup (not backed up) + StartVersion: backupTS, // The backup snapshot time + ... + Path: path, // The path in which backup files are stored + RateLimit: rateLimit, // Backup speed (MB/s) + Concurrency: concurrency, // The number of threads for the backup operation (4 by default) +} +``` + +After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backups some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. + +After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. + +If checksum is enabled when you execute the backup command, BR calculates the checksum of each backed up table for data check. + +### Restoration + +When BR performs the restoration, BR performs the following tasks in order: + +1. It parses the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. + +2. It aggregates the parsed SST file according to the tables and `GroupBy`. + +3. It pre-splits Regions according to the key range of the SST file so that every Region corresponds to at least one SST file. + +4. It traverses every table to be restored and the SST file corresponding to these tables. + +5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. + +After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. + +After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. + +![br-arch](/media/br-arch.png) From 852c40ec9f4d312b00641e19cd08b501e649305a Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Wed, 4 Dec 2019 20:38:36 +0800 Subject: [PATCH 13/20] maintainL refine language --- dev/how-to/maintain/backup-and-restore/br.md | 34 +++++++++---------- v3.1/how-to/maintain/backup-and-restore/br.md | 34 +++++++++---------- 2 files changed, 34 insertions(+), 34 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 9776ed5b141e2..99d7accc073a1 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -20,8 +20,8 @@ In this command, * `backup` is the sub-command of `br`. * `full` is the sub-command of `backup`. -* `-s` or `--storage` specifies the directory in which the backup files are stored. -* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the directory in the local disk for storing backup files. +* `-s` or `--storage` specifies the path in which the backup files are stored. +* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the path in the local disk in which the backup files are stored. * `--pd` is the Placement Driver (PD) service address. * `"${PDIP}:2379"` is the parameter of `--pd`. @@ -58,7 +58,7 @@ For example, set `tikv_gc_life_time` to `720h`: {{< copyable "sql" >}} -``` +```sql mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; ``` @@ -67,7 +67,7 @@ mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. -Use case: Backup all the cluster data to the `/tmp/backup` directory of each TiKV node and write the `backupmeta` file to this directory. +Usage: Backup all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. {{< copyable "shell-regular" >}} @@ -80,7 +80,7 @@ br backup full \ --log-file backupfull.log ``` -In the above `br` command, the `--ratelimit` option specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. The `--concurrency` option sets an upper limit on the number of concurrent executions on each TiKV node. `--log-file` specifies that the BR log is written to the `restorefull.log` file. +In the above `br` command, the `--ratelimit` option specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. The `--concurrency` option sets an upper limit on the number of concurrent executions on each TiKV node. `--log-file` specifies that the BR log is written to the `backupfull.log` file. A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: @@ -98,7 +98,7 @@ Full Backup <---------/................................................> 17.12%. To backup the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. -Use case: Backup the data of the `test.usertable` table to the `/tmp/backup` directory in each TiKV node and write the `backupmeta` file to this directory. +Usage: Backup the data of the `test.usertable` table to the `/tmp/backup` path on each TiKV node and write the `backupmeta` file to this path. {{< copyable "shell-regular" >}} @@ -123,13 +123,13 @@ To restore the cluster data, use the `br restore` command. You can add the `full > **Note:** > -> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. +> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the path specified by `--storage` on each TikV node. ### Restore all backup data To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. -Usage: Restore all the backup data in the `/tmp/backup` directory to the cluster. +Usage: Restore all the backup data in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -143,7 +143,7 @@ br restore full \ In the above command, `--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation. `--log-file` specifies that the BR log is written to the `restorefull.log` file. -A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then, the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then the BR also checks the backup data to ensure data security. ```shell br restore full \ @@ -157,7 +157,7 @@ Full Restore <---------/...............................................> 17.12%. To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. -Usage: Restore a database backed up in the `/tmp/backup` directory to the cluster. +Usage: Restore a database backed up in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -175,7 +175,7 @@ In the above command, `--db` specifies the name of the database to be restored. To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a table backed up in the `/tmp/backup` directory to the cluster. +Usage: Restore a table backed up in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -192,7 +192,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Best practices -- It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. Then you will find it easier to collect and manage backup files. +- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`. In this way, you will find it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. - It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. - To speed up the restoration, use pd-ctl to remove the schedulers related to scheduling before the restoration and add back these removed schedulers after the restoration. @@ -221,7 +221,7 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - TiDB cannot perform the backup operation when executing DDL operations. -- Currently TiDB does not support the backup and restoration of the partition table. +- Currently TiDB does not support backing up and restoring partition tables. - Currently you can perform the restoration only on new clusters. ## Examples @@ -288,7 +288,7 @@ Then execute the following command to backup all the cluster data: {{< copyable "shell-regular" >}} -``` +```shell bin/br backup full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file backup.log ``` @@ -305,7 +305,7 @@ Then execute the following command to restore all the cluster data: {{< copyable "shell-regular" >}} -``` +```shell bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restore.log ``` @@ -316,7 +316,7 @@ bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restor ## Working principles -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. ### Backup @@ -359,7 +359,7 @@ If checksum is enabled when you execute the backup command, BR calculates the ch When BR performs the restoration, BR performs the following tasks in order: -1. It parses the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. +1. It parses the backupMeta file in the backup path, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. 2. It aggregates the parsed SST file according to the tables and `GroupBy`. diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 3dbbdf573c16d..1ef73537a2c93 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -20,8 +20,8 @@ In this command, * `backup` is the sub-command of `br`. * `full` is the sub-command of `backup`. -* `-s` or `--storage` specifies the directory in which the backup files are stored. -* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the directory in the local disk for storing backup files. +* `-s` or `--storage` specifies the path in which the backup files are stored. +* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the path in the local disk in which the backup files are stored. * `--pd` is the Placement Driver (PD) service address. * `"${PDIP}:2379"` is the parameter of `--pd`. @@ -58,7 +58,7 @@ For example, set `tikv_gc_life_time` to `720h`: {{< copyable "sql" >}} -``` +```sql mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; ``` @@ -67,7 +67,7 @@ mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. -Use case: Backup all the cluster data to the `/tmp/backup` directory of each TiKV node and write the `backupmeta` file to this directory. +Usage: Backup all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. {{< copyable "shell-regular" >}} @@ -80,7 +80,7 @@ br backup full \ --log-file backupfull.log ``` -In the above `br` command, the `--ratelimit` option specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. The `--concurrency` option sets an upper limit on the number of concurrent executions on each TiKV node. `--log-file` specifies that the BR log is written to the `restorefull.log` file. +In the above `br` command, the `--ratelimit` option specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. The `--concurrency` option sets an upper limit on the number of concurrent executions on each TiKV node. `--log-file` specifies that the BR log is written to the `backupfull.log` file. A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: @@ -98,7 +98,7 @@ Full Backup <---------/................................................> 17.12%. To backup the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. -Use case: Backup the data of the `test.usertable` table to the `/tmp/backup` directory in each TiKV node and write the `backupmeta` file to this directory. +Usage: Backup the data of the `test.usertable` table to the `/tmp/backup` path on each TiKV node and write the `backupmeta` file to this path. {{< copyable "shell-regular" >}} @@ -123,13 +123,13 @@ To restore the cluster data, use the `br restore` command. You can add the `full > **Note:** > -> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the directory specified by `--storage` on each TikV node. +> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the path specified by `--storage` on each TikV node. ### Restore all backup data To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. -Usage: Restore all the backup data in the `/tmp/backup` directory to the cluster. +Usage: Restore all the backup data in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -143,7 +143,7 @@ br restore full \ In the above command, `--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation. `--log-file` specifies that the BR log is written to the `restorefull.log` file. -A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then, the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then the BR also checks the backup data to ensure data security. ```shell br restore full \ @@ -157,7 +157,7 @@ Full Restore <---------/...............................................> 17.12%. To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. -Usage: Restore a database backed up in the `/tmp/backup` directory to the cluster. +Usage: Restore a database backed up in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -175,7 +175,7 @@ In the above command, `--db` specifies the name of the database to be restored. To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a table backed up in the `/tmp/backup` directory to the cluster. +Usage: Restore a table backed up in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -192,7 +192,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Best practices -- It is recommended that you mount a shared storage (for example, NFS) on the backup directory specified by `-s`. Then you will find it easier to collect and manage backup files. +- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`. In this way, you will find it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. - It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. - To speed up the restoration, use pd-ctl to remove the schedulers related to scheduling before the restoration and add back these removed schedulers after the restoration. @@ -221,7 +221,7 @@ In the above command, `--table` specifies the name of the table to be restored. - BR only supports TiDB v3.1 or later versions. - TiDB cannot perform the backup operation when executing DDL operations. -- Currently TiDB does not support the backup and restoration of the partition table. +- Currently TiDB does not support backing up and restoring partition tables. - Currently you can perform the restoration only on new clusters. ## Examples @@ -288,7 +288,7 @@ Then execute the following command to backup all the cluster data: {{< copyable "shell-regular" >}} -``` +```shell bin/br backup full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file backup.log ``` @@ -305,7 +305,7 @@ Then execute the following command to restore all the cluster data: {{< copyable "shell-regular" >}} -``` +```shell bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restore.log ``` @@ -316,7 +316,7 @@ bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restor ## Working principles -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a directory in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. ### Backup @@ -359,7 +359,7 @@ If checksum is enabled when you execute the backup command, BR calculates the ch When BR performs the restoration, BR performs the following tasks in order: -1. It parses the backupMeta file in the backup directory, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. +1. It parses the backupMeta file in the backup path, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. 2. It aggregates the parsed SST file according to the tables and `GroupBy`. From e2a6c5b1362be911fe9280c0ac791182d8313431 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Thu, 5 Dec 2019 11:15:09 +0800 Subject: [PATCH 14/20] maintain: minor language change --- dev/how-to/maintain/backup-and-restore/br.md | 8 ++++---- v3.1/how-to/maintain/backup-and-restore/br.md | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 99d7accc073a1..c01a7714d2068 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -220,9 +220,9 @@ In the above command, `--table` specifies the name of the table to be restored. ## Usage restrictions - BR only supports TiDB v3.1 or later versions. -- TiDB cannot perform the backup operation when executing DDL operations. +- TiDB cannot perform backup operation when executing DDL operations. - Currently TiDB does not support backing up and restoring partition tables. -- Currently you can perform the restoration only on new clusters. +- Currently you can perform restoration only on new clusters. ## Examples @@ -320,7 +320,7 @@ BR sends the backup and restoration commands to each TiKV node. After receiving ### Backup -When BR performs a backup operation, it first obtains the following information from the PD: +When BR performs a backup operation, it first obtains the following information from PD: - The current TS (timestamp) as the time of the backup snapshot - The TiKV node information of the current cluster @@ -369,7 +369,7 @@ When BR performs the restoration, BR performs the following tasks in order: 5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. -After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. +After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file is deleted asynchronously. After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 1ef73537a2c93..06274a0c4b678 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -220,9 +220,9 @@ In the above command, `--table` specifies the name of the table to be restored. ## Usage restrictions - BR only supports TiDB v3.1 or later versions. -- TiDB cannot perform the backup operation when executing DDL operations. +- TiDB cannot perform backup operation when executing DDL operations. - Currently TiDB does not support backing up and restoring partition tables. -- Currently you can perform the restoration only on new clusters. +- Currently you can perform restoration only on new clusters. ## Examples @@ -320,7 +320,7 @@ BR sends the backup and restoration commands to each TiKV node. After receiving ### Backup -When BR performs a backup operation, it first obtains the following information from the PD: +When BR performs a backup operation, it first obtains the following information from PD: - The current TS (timestamp) as the time of the backup snapshot - The TiKV node information of the current cluster @@ -369,7 +369,7 @@ When BR performs the restoration, BR performs the following tasks in order: 5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. -After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file will be deleted asynchronously. +After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file is deleted asynchronously. After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. From c6ab9e4cdf3850fa98a29e3eb9473acdfe50bf38 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 6 Dec 2019 15:37:18 +0800 Subject: [PATCH 15/20] address comments --- dev/how-to/maintain/backup-and-restore/br.md | 112 ++++++++++++------- 1 file changed, 71 insertions(+), 41 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index c01a7714d2068..f62af5e53840f 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -1,56 +1,64 @@ --- title: Use BR to Backup and Restore Cluster Data -summary: Learn how to backup and restore the data of the TiDB cluster using BR. +summary: Learn how to backup and restore data of the TiDB cluster using BR. category: how-to --- # Use BR to Backup and Restore Cluster Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the command line, offers detailed use cases, best practices, usage restrictions, and introduces the working principle of BR. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the BR command line, detailed use examples, best practices, restrictions, and introduces the working principles of BR. ## Command-line description -A `br` command consists of sub-commands, options and parameters. The sub-command is the characters without `-` or `--`. The option is the characters that start with `-` or `--`. The parameter is the characters that immediately follow and pass to the sub-command or the option. +A `br` command consists of sub-commands, options, and parameters. + +* Sub-command: the characters without `-` or `--`. +* Option: the characters that start with `-` or `--`. +* Parameter: the characters that immediately follow behind and are passed to the sub-command or the option. This is a complete `br` command: -`br backup full --pd "${PDIP}:2379" -s "local:///tmp/backup"` +{{< copyable "shell-regular" >}} -In this command, +```shell +br backup full --pd "${PDIP}:2379" -s "local:///tmp/backup" +``` -* `backup` is the sub-command of `br`. -* `full` is the sub-command of `backup`. -* `-s` or `--storage` specifies the path in which the backup files are stored. -* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the path in the local disk in which the backup files are stored. -* `--pd` is the Placement Driver (PD) service address. -* `"${PDIP}:2379"` is the parameter of `--pd`. +Explanations for the above command are as follows: + +* `backup`: the sub-command of `br`. +* `full`: the sub-command of `backup`. +* `-s` (or `--storage`): the option that specifies the path where the backup files are stored. +* `"local:///tmp/backup"`: the parameter of `-s`. `/tmp/backup` is the path in the local disk where the backup files are stored. +* `--pd`: the option that specifies the Placement Driver (PD) service address. +* `"${PDIP}:2379"`: the parameter of `--pd`. ### Sub-commands A `br` command consists of multiple layers of sub-commands. Currently, BR has the following three sub-commands: -* `br backup` is used to backup the data of the TiDB cluster. -* `br restore` is used to restore the data of the TiDB cluster. -* `br version` is used to check the version of BR. +* `br backup`: used to back up the data of the TiDB cluster. +* `br restore`: used to restore the data of the TiDB cluster. +* `br version`: used to check the version of BR. Each of the above three sub-commands might include the following three sub-commands to specify the scope of an operation: -* `full` is used to backup or restore all the cluster data. -* `db` is used to restore the specified database of the cluster. -* `table` is used to backup or restore a single table in the specified database of the cluster. +* `full`: used to back up or restore all the cluster data. +* `db`: used to restore the specified database of the cluster. +* `table`: used to back up or restore a single table in the specified database of the cluster. ### Common options -* `--pd` is for connection, specifying the PD server address. For example, `"${PDIP}:2379"`. -* `-h`/`--help` is used to get help on all sub-commands. For example, `br backup --help`. -* `--ca` specifies the path to the trusted CA certificate in the PEM format. -* `--cert` specifies the path to the SSL certificate in the PEM format. -* `--key` specifies the path to the SSL certificate key in the PEM format. -* `--status-addr` specifies the listening address through which BR provides statistics to Prometheus. +* `--pd`: used for connection, specifying the PD server address. For example, `"${PDIP}:2379"`. +* `-h` (or `--help`): used to get help on all sub-commands. For example, `br backup --help`. +* `--ca`: specifies the path to the trusted CA certificate in the PEM format. +* `--cert`: specifies the path to the SSL certificate in the PEM format. +* `--key`: specifies the path to the SSL certificate key in the PEM format. +* `--status-addr`: specifies the listening address through which BR provides statistics to Prometheus. -## Backup cluster data +## Back up cluster data -To backup the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table. +To back up the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table. If the backup time might exceed the [`tikv_gc_life_time`](/dev/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. @@ -63,11 +71,13 @@ mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; ``` -### Backup all cluster data +### Back up all cluster data + +To back up all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. -To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. +**Usage example:** -Usage: Backup all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. +Back up all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. {{< copyable "shell-regular" >}} @@ -80,7 +90,11 @@ br backup full \ --log-file backupfull.log ``` -In the above `br` command, the `--ratelimit` option specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. The `--concurrency` option sets an upper limit on the number of concurrent executions on each TiKV node. `--log-file` specifies that the BR log is written to the `backupfull.log` file. +Explanations for some options in the above command are as follows: + +* `--ratelimit`: specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. +* `--concurrency`: sets an upper limit on the number of concurrent executions on each TiKV node. +* `--log-file`: specifies that the BR log is written to the `backupfull.log` file. A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: @@ -94,11 +108,13 @@ br backup full \ Full Backup <---------/................................................> 17.12%. ``` -### Backup single table +### Back up single table -To backup the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. +To back up the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. -Usage: Backup the data of the `test.usertable` table to the `/tmp/backup` path on each TiKV node and write the `backupmeta` file to this path. +**Usage example:** + +Back up the data of the `test.usertable` table to the `/tmp/backup` path on each TiKV node and write the `backupmeta` file to this path. {{< copyable "shell-regular" >}} @@ -113,7 +129,12 @@ br backup table \ --log-file backuptable.log ``` -The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. For the meanings of other options, see [Backup all cluster data](#backup-all-cluster-data). +The `table` sub-command has two options: `--db` and `--table`: + +* `--db`: specifies the database name +* `--table`: specifies the table name. + +For the meanings of other options, see [Back up all cluster data](#back-up-all-cluster-data). A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. @@ -129,7 +150,9 @@ To restore the cluster data, use the `br restore` command. You can add the `full To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. -Usage: Restore all the backup data in the `/tmp/backup` path to the cluster. +**Usage example:** + +Restore all the backup data in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -141,7 +164,10 @@ br restore full \ --log-file restorefull.log ``` -In the above command, `--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation. `--log-file` specifies that the BR log is written to the `restorefull.log` file. +Explanations for some options in the above command are as follows: + +* `--concurrency`: specifies how many sub-tasks can be performed concurrently in a restoration operation. +* `--log-file`: specifies that the BR log is written to the `restorefull.log` file. A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then the BR also checks the backup data to ensure data security. @@ -157,7 +183,9 @@ Full Restore <---------/...............................................> 17.12%. To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. -Usage: Restore a database backed up in the `/tmp/backup` path to the cluster. +**Usage example:** + +Restore a database backed up in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -175,7 +203,9 @@ In the above command, `--db` specifies the name of the database to be restored. To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a table backed up in the `/tmp/backup` path to the cluster. +**Usage example:** + +Restore a table backed up in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -188,7 +218,7 @@ br restore table \ --log-file restorefull.log ``` -In the above command, `--table` specifies the name of the table to be restored. For the meanings of other options, see [Restore a database](#restore-a-database). +In the above command, `--table` specifies the name of the table to be restored. For the meanings of other options, see [Restore all backup data](#restore-all-backup-data) and [Restore a database](#restore-a-database). ## Best practices @@ -226,7 +256,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Examples -This section shows how to backup and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. +This section shows how to back up and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. ### Data size and machine configuration @@ -284,7 +314,7 @@ Before the backup operation, make sure the following things are done: - `tikv_gc_life_time` is set to a larger value so that the backup operation will not be interrupted by data loss. - No DDL statement is executed on TiDB. -Then execute the following command to backup all the cluster data: +Then execute the following command to back up all the cluster data: {{< copyable "shell-regular" >}} @@ -349,7 +379,7 @@ backup.BackupRequest{ } ``` -After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backups some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. +After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backs up some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. From 7c06d381bcd266cb6b6dd64eb1b7888ae125d0ae Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 6 Dec 2019 17:03:03 +0800 Subject: [PATCH 16/20] address comments --- dev/how-to/maintain/backup-and-restore/br.md | 56 ++++++++++---------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index f62af5e53840f..33c0b3cd03035 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -1,10 +1,10 @@ --- -title: Use BR to Backup and Restore Cluster Data -summary: Learn how to backup and restore data of the TiDB cluster using BR. +title: Use BR to Back up and Restore Data +summary: Learn how to back up and restore data of the TiDB cluster using BR. category: how-to --- -# Use BR to Backup and Restore Cluster Data +# Use BR to Back up and Restore Data Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the BR command line, detailed use examples, best practices, restrictions, and introduces the working principles of BR. @@ -41,7 +41,7 @@ A `br` command consists of multiple layers of sub-commands. Currently, BR has th * `br restore`: used to restore the data of the TiDB cluster. * `br version`: used to check the version of BR. -Each of the above three sub-commands might include the following three sub-commands to specify the scope of an operation: +Each of the above three sub-commands might still include the following three sub-commands to specify the scope of an operation: * `full`: used to back up or restore all the cluster data. * `db`: used to restore the specified database of the cluster. @@ -71,7 +71,7 @@ mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; ``` -### Back up all cluster data +### Back up all the cluster data To back up all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. @@ -93,10 +93,10 @@ br backup full \ Explanations for some options in the above command are as follows: * `--ratelimit`: specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. -* `--concurrency`: sets an upper limit on the number of concurrent executions on each TiKV node. -* `--log-file`: specifies that the BR log is written to the `backupfull.log` file. +* `--concurrency`: sets an upper limit on the number of concurrent operations on each TiKV node. +* `--log-file`: specifies writing the BR log to the `backupfull.log` file. -A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: +A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data safety. The progress bar is displayed as follows: ```shell br backup full \ @@ -108,7 +108,7 @@ br backup full \ Full Backup <---------/................................................> 17.12%. ``` -### Back up single table +### Back up data of a single table To back up the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. @@ -129,14 +129,14 @@ br backup table \ --log-file backuptable.log ``` -The `table` sub-command has two options: `--db` and `--table`: +The `table` sub-command has two options: * `--db`: specifies the database name * `--table`: specifies the table name. -For the meanings of other options, see [Back up all cluster data](#back-up-all-cluster-data). +For descriptions of other options, see [Back up all cluster data](#back-up-all-cluster-data). -A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data safety. ## Restore cluster data @@ -144,9 +144,9 @@ To restore the cluster data, use the `br restore` command. You can add the `full > **Note:** > -> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the path specified by `--storage` on each TikV node. +> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the path specified by `--storage` on each TiKV node. -### Restore all backup data +### Restore all the backup data To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. @@ -167,9 +167,9 @@ br restore full \ Explanations for some options in the above command are as follows: * `--concurrency`: specifies how many sub-tasks can be performed concurrently in a restoration operation. -* `--log-file`: specifies that the BR log is written to the `restorefull.log` file. +* `--log-file`: specifies writing the BR log to the `restorefull.log` file. -A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then the BR also checks the backup data to ensure data safety. ```shell br restore full \ @@ -197,7 +197,7 @@ br restore db \ --log-file restorefull.log ``` -In the above command, `--db` specifies the name of the database to be restored. For the meanings of other options, see [Restore all backup data](#restore-all-backup-data). +In the above command, `--db` specifies the name of the database to be restored. For descriptions of other options, see [Restore all backup data](#restore-all-backup-data). ### Restore a table @@ -218,13 +218,13 @@ br restore table \ --log-file restorefull.log ``` -In the above command, `--table` specifies the name of the table to be restored. For the meanings of other options, see [Restore all backup data](#restore-all-backup-data) and [Restore a database](#restore-a-database). +In the above command, `--table` specifies the name of the table to be restored. For descriptions of other options, see [Restore all backup data](#restore-all-backup-data) and [Restore a database](#restore-a-database). ## Best practices -- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`. In this way, you will find it easier to collect and manage backup files. +- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`, to make it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. -- It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. +- It is recommended that you perform the backup operation during off-peak hours to minimize the impact on applications. - To speed up the restoration, use pd-ctl to remove the schedulers related to scheduling before the restoration and add back these removed schedulers after the restoration. Remove schedulers: @@ -249,18 +249,18 @@ In the above command, `--table` specifies the name of the table to be restored. ## Usage restrictions -- BR only supports TiDB v3.1 or later versions. +- BR only supports TiDB v3.1 and later versions. - TiDB cannot perform backup operation when executing DDL operations. -- Currently TiDB does not support backing up and restoring partition tables. -- Currently you can perform restoration only on new clusters. +- Currently, TiDB does not support backing up and restoring partitioned tables. +- Currently, you can perform restoration only on new clusters. ## Examples This section shows how to back up and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. -### Data size and machine configuration +### Data volume and machine configuration -Suppose that the backup and restoration operations are performed on 10 tables in the TiKV cluster, each table with 5 million rows of data. The total data size is 35 GB. +Suppose that the backup and restoration operations are performed on 10 tables in the TiKV cluster, each table with 5 million rows of data. The total data volume is 35 GB. ```sql MySQL [sbtest]> show tables; @@ -309,10 +309,10 @@ Suppose that 4 TiKV nodes is used, each with the following configuration: ### Backup -Before the backup operation, make sure the following things are done: +Before the backup operation, check the following two items: -- `tikv_gc_life_time` is set to a larger value so that the backup operation will not be interrupted by data loss. -- No DDL statement is executed on TiDB. +- You have set `tikv_gc_life_time` set to a larger value so that the backup operation will not be interrupted because of data loss. +- No DDL statement is being executed on the TiDB cluster. Then execute the following command to back up all the cluster data: From 9877d87b44fa46c02fd1bc4b4aa023ac7fd61082 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 6 Dec 2019 17:36:32 +0800 Subject: [PATCH 17/20] address comments --- dev/TOC.md | 2 +- dev/how-to/maintain/backup-and-restore/br.md | 32 ++-- .../backup-and-restore/mydumper-loader.md | 8 +- v3.1/TOC.md | 2 +- v3.1/how-to/maintain/backup-and-restore/br.md | 178 ++++++++++-------- .../backup-and-restore/mydumper-loader.md | 8 +- 6 files changed, 130 insertions(+), 100 deletions(-) diff --git a/dev/TOC.md b/dev/TOC.md index 22e5f7ff2b588..f1c426930a071 100644 --- a/dev/TOC.md +++ b/dev/TOC.md @@ -84,7 +84,7 @@ + Maintain - [Common Ansible Operations](/dev/how-to/deploy/orchestrated/ansible-operations.md) + Backup and Restore - - [Use Mydumper/Loader](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md) + - [Use `mydumper` and `loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md) - [Use BR](/dev/how-to/maintain/backup-and-restore/br.md) - [Identify Slow Queries](/dev/how-to/maintain/identify-slow-queries.md) + Scale diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 33c0b3cd03035..35b01eced83c3 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -6,7 +6,7 @@ category: how-to # Use BR to Back up and Restore Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the BR command line, detailed use examples, best practices, restrictions, and introduces the working principles of BR. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/dev/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the BR command line, detailed use examples, best practices, restrictions, and introduces the implementation principles of BR. ## Command-line description @@ -344,11 +344,11 @@ bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restor [INFO] [client.go:435] ["Restore Checksum"] [take=6.385818026s] ``` -## Working principles +## Implementation principles -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read during the restoration. -### Backup +### Backup principle When BR performs a backup operation, it first obtains the following information from PD: @@ -357,9 +357,9 @@ When BR performs a backup operation, it first obtains the following information According to these information, BR starts a TiDB instance internally to obtain the database or table information corresponding to the TS, and filters out the system databases (`information_schema`, `performance_schema`, `mysql`) at the same time. -According to the backup sub-command, two types of backup logic are available: +According to the backup sub-command, BR adopts the following two types of backup logic: -- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to every table. +- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to each table. - Single table backup: BR constructs the KV range to be backed up according a single table. Finally, BR collects the KV range to be backed up and sends the complete backup request to the TiKV node of the cluster. @@ -368,39 +368,39 @@ The structure of the request: ``` backup.BackupRequest{ - ClusterId: clusterID, // The cluster ID + ClusterId: clusterID, // The cluster ID. StartKey: startKey, // The starting key of the backup (backed up) EndKey: endKey, // The ending key of the backup (not backed up) StartVersion: backupTS, // The backup snapshot time ... - Path: path, // The path in which backup files are stored + Path: path, // The path where backup files are stored RateLimit: rateLimit, // Backup speed (MB/s) Concurrency: concurrency, // The number of threads for the backup operation (4 by default) } ``` -After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backs up some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. +After receiving the backup request, the TiKV node traverses all Region leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backs up some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. -After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. +After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the `backupMeta` file which is used for restoration. If checksum is enabled when you execute the backup command, BR calculates the checksum of each backed up table for data check. -### Restoration +### Restoration principle -When BR performs the restoration, BR performs the following tasks in order: +During the data restoration process, BR performs the following tasks in order: -1. It parses the backupMeta file in the backup path, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. +1. It parses the `backupMeta` file in the backup path, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. -2. It aggregates the parsed SST file according to the tables and `GroupBy`. +2. It aggregates the parsed SST files according to the tables and `GroupBy`. 3. It pre-splits Regions according to the key range of the SST file so that every Region corresponds to at least one SST file. -4. It traverses every table to be restored and the SST file corresponding to these tables. +4. It traverses each table to be restored and the SST file corresponding to each tables. 5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file is deleted asynchronously. -After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. +After the restoration operation is complete, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. ![br-arch](/media/br-arch.png) diff --git a/dev/how-to/maintain/backup-and-restore/mydumper-loader.md b/dev/how-to/maintain/backup-and-restore/mydumper-loader.md index 9a549219a6c01..4e2ce209f08c0 100644 --- a/dev/how-to/maintain/backup-and-restore/mydumper-loader.md +++ b/dev/how-to/maintain/backup-and-restore/mydumper-loader.md @@ -1,13 +1,13 @@ --- -title: Use `mydumper`/`loader` to Backup and Restore -summary: Learn how to backup and restore the data of TiDB using `mydumper`/`loader`. +title: Use `mydumper`and `loader` to Back up and Restore Data +summary: Learn how to back up and restore the data of TiDB using `mydumper` and `loader`. category: how-to aliases: ['/docs/dev/how-to/maintain/backup-and-restore/'] --- -# Use `mydumper`/`loader` to Backup and Restore +# Use `mydumper` and `loader` to Back up and Restore Data -This document describes how to back up and restore the data of TiDB using `mydumper`/`loader`. Currently, this document only covers full backup and restoration. +This document describes how to back up and restore the data of TiDB using `mydumper` and `loader`. Currently, this document only covers full backup and restoration. Suppose that the TiDB service information is as follows: diff --git a/v3.1/TOC.md b/v3.1/TOC.md index 0bdcccac60c13..08de2f1f1c717 100644 --- a/v3.1/TOC.md +++ b/v3.1/TOC.md @@ -83,7 +83,7 @@ + Maintain - [Common Ansible Operations](/v3.1/how-to/deploy/orchestrated/ansible-operations.md) + Backup and Restore - - [Use Mydumper/Loader](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md) + - [Use `mydumper` and `loader`](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md) - [Use BR](/v3.1/how-to/maintain/backup-and-restore/br.md) - [Identify Slow Queries](/v3.1/how-to/maintain/identify-slow-queries.md) + Scale diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 06274a0c4b678..ea85566e92f7b 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -1,56 +1,64 @@ --- -title: Use BR to Backup and Restore Cluster Data -summary: Learn how to backup and restore the data of the TiDB cluster using BR. +title: Use BR to Back up and Restore Data +summary: Learn how to back up and restore data of the TiDB cluster using BR. category: how-to --- -# Use BR to Backup and Restore Cluster Data +# Use BR to Back up and Restore Data -Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the command line, offers detailed use cases, best practices, usage restrictions, and introduces the working principle of BR. +Backup & Restore (BR) is a command-line tool for distributed backup and restoration of the TiDB cluster data. Compared with [`mydumper`/`loader`](/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md), BR is more suitable for scenarios of huge data volume. This document describes the BR command line, detailed use examples, best practices, restrictions, and introduces the implementation principles of BR. ## Command-line description -A `br` command consists of sub-commands, options and parameters. The sub-command is the characters without `-` or `--`. The option is the characters that start with `-` or `--`. The parameter is the characters that immediately follow and pass to the sub-command or the option. +A `br` command consists of sub-commands, options, and parameters. + +* Sub-command: the characters without `-` or `--`. +* Option: the characters that start with `-` or `--`. +* Parameter: the characters that immediately follow behind and are passed to the sub-command or the option. This is a complete `br` command: -`br backup full --pd "${PDIP}:2379" -s "local:///tmp/backup"` +{{< copyable "shell-regular" >}} + +```shell +br backup full --pd "${PDIP}:2379" -s "local:///tmp/backup" +``` -In this command, +Explanations for the above command are as follows: -* `backup` is the sub-command of `br`. -* `full` is the sub-command of `backup`. -* `-s` or `--storage` specifies the path in which the backup files are stored. -* `"local:///tmp/backup"` is the parameter of `-s`. `/tmp/backup` is the path in the local disk in which the backup files are stored. -* `--pd` is the Placement Driver (PD) service address. -* `"${PDIP}:2379"` is the parameter of `--pd`. +* `backup`: the sub-command of `br`. +* `full`: the sub-command of `backup`. +* `-s` (or `--storage`): the option that specifies the path where the backup files are stored. +* `"local:///tmp/backup"`: the parameter of `-s`. `/tmp/backup` is the path in the local disk where the backup files are stored. +* `--pd`: the option that specifies the Placement Driver (PD) service address. +* `"${PDIP}:2379"`: the parameter of `--pd`. ### Sub-commands A `br` command consists of multiple layers of sub-commands. Currently, BR has the following three sub-commands: -* `br backup` is used to backup the data of the TiDB cluster. -* `br restore` is used to restore the data of the TiDB cluster. -* `br version` is used to check the version of BR. +* `br backup`: used to back up the data of the TiDB cluster. +* `br restore`: used to restore the data of the TiDB cluster. +* `br version`: used to check the version of BR. -Each of the above three sub-commands might include the following three sub-commands to specify the scope of an operation: +Each of the above three sub-commands might still include the following three sub-commands to specify the scope of an operation: -* `full` is used to backup or restore all the cluster data. -* `db` is used to restore the specified database of the cluster. -* `table` is used to backup or restore a single table in the specified database of the cluster. +* `full`: used to back up or restore all the cluster data. +* `db`: used to restore the specified database of the cluster. +* `table`: used to back up or restore a single table in the specified database of the cluster. ### Common options -* `--pd` is for connection, specifying the PD server address. For example, `"${PDIP}:2379"`. -* `-h`/`--help` is used to get help on all sub-commands. For example, `br backup --help`. -* `--ca` specifies the path to the trusted CA certificate in the PEM format. -* `--cert` specifies the path to the SSL certificate in the PEM format. -* `--key` specifies the path to the SSL certificate key in the PEM format. -* `--status-addr` specifies the listening address through which BR provides statistics to Prometheus. +* `--pd`: used for connection, specifying the PD server address. For example, `"${PDIP}:2379"`. +* `-h` (or `--help`): used to get help on all sub-commands. For example, `br backup --help`. +* `--ca`: specifies the path to the trusted CA certificate in the PEM format. +* `--cert`: specifies the path to the SSL certificate in the PEM format. +* `--key`: specifies the path to the SSL certificate key in the PEM format. +* `--status-addr`: specifies the listening address through which BR provides statistics to Prometheus. -## Backup cluster data +## Back up cluster data -To backup the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table. +To back up the cluster data, use the `br backup` command. You can add the `full` or `table` sub-command to specify the scope of your backup operation: the whole cluster or a single table. If the backup time might exceed the [`tikv_gc_life_time`](/v3.1/reference/garbage-collection/configuration.md#tikv_gc_life_time) configuration which is `10m0s` by default, increase the value of this configuration. @@ -63,11 +71,13 @@ mysql -h${TiDBIP} -P4000 -u${TIDB_USER} ${password_str} -Nse \ "update mysql.tidb set variable_value='720h' where variable_name='tikv_gc_life_time'"; ``` -### Backup all cluster data +### Back up all the cluster data + +To back up all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. -To backup all the cluster data, execute the `br backup full` command. To get help on this command, execute `br backup full -h` or `br backup full --help`. +**Usage example:** -Usage: Backup all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. +Back up all the cluster data to the `/tmp/backup` path of each TiKV node and write the `backupmeta` file to this path. {{< copyable "shell-regular" >}} @@ -80,9 +90,13 @@ br backup full \ --log-file backupfull.log ``` -In the above `br` command, the `--ratelimit` option specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. The `--concurrency` option sets an upper limit on the number of concurrent executions on each TiKV node. `--log-file` specifies that the BR log is written to the `backupfull.log` file. +Explanations for some options in the above command are as follows: -A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. The progress bar is displayed as follows: +* `--ratelimit`: specifies the maximum speed at which a backup operation is performed (MiB/s) on each TiKV node. +* `--concurrency`: sets an upper limit on the number of concurrent operations on each TiKV node. +* `--log-file`: specifies writing the BR log to the `backupfull.log` file. + +A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data safety. The progress bar is displayed as follows: ```shell br backup full \ @@ -94,11 +108,13 @@ br backup full \ Full Backup <---------/................................................> 17.12%. ``` -### Backup single table +### Back up data of a single table + +To back up the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. -To backup the data of a single table in the cluster, execute the `br backup table` command. To get help on this command, execute `br backup table -h` or `br backup table --help`. +**Usage example:** -Usage: Backup the data of the `test.usertable` table to the `/tmp/backup` path on each TiKV node and write the `backupmeta` file to this path. +Back up the data of the `test.usertable` table to the `/tmp/backup` path on each TiKV node and write the `backupmeta` file to this path. {{< copyable "shell-regular" >}} @@ -113,9 +129,14 @@ br backup table \ --log-file backuptable.log ``` -The `table` sub-command has two options: `--db` and `--table`. `--db` specifies the database name and `--table` specifies the table name. For the meanings of other options, see [Backup all cluster data](#backup-all-cluster-data). +The `table` sub-command has two options: -A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data security. +* `--db`: specifies the database name +* `--table`: specifies the table name. + +For descriptions of other options, see [Back up all cluster data](#back-up-all-cluster-data). + +A progress bar is displayed in the terminal during the backup operation. When the progress bar advances to 100%, the backup is complete. Then the BR also checks the backup data to ensure data safety. ## Restore cluster data @@ -123,13 +144,15 @@ To restore the cluster data, use the `br restore` command. You can add the `full > **Note:** > -> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the path specified by `--storage` on each TikV node. +> If the backed up cluster does not have a network storage, before the restoration, copy the backup SST files to the path specified by `--storage` on each TiKV node. -### Restore all backup data +### Restore all the backup data To restore all the backup data to the cluster, execute the `br restore full` command. To get help on this command, execute `br restore full -h` or `br restore full --help`. -Usage: Restore all the backup data in the `/tmp/backup` path to the cluster. +**Usage example:** + +Restore all the backup data in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -141,9 +164,12 @@ br restore full \ --log-file restorefull.log ``` -In the above command, `--concurrency` specifies how many sub-tasks can be performed concurrently in a restoration operation. `--log-file` specifies that the BR log is written to the `restorefull.log` file. +Explanations for some options in the above command are as follows: + +* `--concurrency`: specifies how many sub-tasks can be performed concurrently in a restoration operation. +* `--log-file`: specifies writing the BR log to the `restorefull.log` file. -A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then the BR also checks the backup data to ensure data security. +A progress bar is displayed in the terminal during the restoration. When the progress bar advances to 100%, the restoration is complete. Then the BR also checks the backup data to ensure data safety. ```shell br restore full \ @@ -157,7 +183,9 @@ Full Restore <---------/...............................................> 17.12%. To restore a database to the cluster, execute the `br restore db` command. To get help on this command, execute `br restore db -h` or `br restore db --help`. -Usage: Restore a database backed up in the `/tmp/backup` path to the cluster. +**Usage example:** + +Restore a database backed up in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -169,13 +197,15 @@ br restore db \ --log-file restorefull.log ``` -In the above command, `--db` specifies the name of the database to be restored. For the meanings of other options, see [Restore all backup data](#restore-all-backup-data). +In the above command, `--db` specifies the name of the database to be restored. For descriptions of other options, see [Restore all backup data](#restore-all-backup-data). ### Restore a table To restore a single table to the cluster, execute the `br restore table` command. To get help on this command, execute `br restore table -h` or `br restore table --help`. -Usage: Restore a table backed up in the `/tmp/backup` path to the cluster. +**Usage example:** + +Restore a table backed up in the `/tmp/backup` path to the cluster. {{< copyable "shell-regular" >}} @@ -188,13 +218,13 @@ br restore table \ --log-file restorefull.log ``` -In the above command, `--table` specifies the name of the table to be restored. For the meanings of other options, see [Restore a database](#restore-a-database). +In the above command, `--table` specifies the name of the table to be restored. For descriptions of other options, see [Restore all backup data](#restore-all-backup-data) and [Restore a database](#restore-a-database). ## Best practices -- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`. In this way, you will find it easier to collect and manage backup files. +- It is recommended that you mount a shared storage (for example, NFS) on the backup path specified by `-s`, to make it easier to collect and manage backup files. - It is recommended that you use a storage hardware with high throughput, because the throughput of a storage hardware limits the backup and restoration speed. -- It is recommended that you perform the backup operation in a low-peak time to minimize the impact on the application. +- It is recommended that you perform the backup operation during off-peak hours to minimize the impact on applications. - To speed up the restoration, use pd-ctl to remove the schedulers related to scheduling before the restoration and add back these removed schedulers after the restoration. Remove schedulers: @@ -219,18 +249,18 @@ In the above command, `--table` specifies the name of the table to be restored. ## Usage restrictions -- BR only supports TiDB v3.1 or later versions. +- BR only supports TiDB v3.1 and later versions. - TiDB cannot perform backup operation when executing DDL operations. -- Currently TiDB does not support backing up and restoring partition tables. -- Currently you can perform restoration only on new clusters. +- Currently, TiDB does not support backing up and restoring partitioned tables. +- Currently, you can perform restoration only on new clusters. ## Examples -This section shows how to backup and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. +This section shows how to back up and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. -### Data size and machine configuration +### Data volume and machine configuration -Suppose that the backup and restoration operations are performed on 10 tables in the TiKV cluster, each table with 5 million rows of data. The total data size is 35 GB. +Suppose that the backup and restoration operations are performed on 10 tables in the TiKV cluster, each table with 5 million rows of data. The total data volume is 35 GB. ```sql MySQL [sbtest]> show tables; @@ -279,12 +309,12 @@ Suppose that 4 TiKV nodes is used, each with the following configuration: ### Backup -Before the backup operation, make sure the following things are done: +Before the backup operation, check the following two items: -- `tikv_gc_life_time` is set to a larger value so that the backup operation will not be interrupted by data loss. -- No DDL statement is executed on TiDB. +- You have set `tikv_gc_life_time` set to a larger value so that the backup operation will not be interrupted because of data loss. +- No DDL statement is being executed on the TiDB cluster. -Then execute the following command to backup all the cluster data: +Then execute the following command to back up all the cluster data: {{< copyable "shell-regular" >}} @@ -314,11 +344,11 @@ bin/br restore full -s local:///tmp/backup --pd "${PDIP}:2379" --log-file restor [INFO] [client.go:435] ["Restore Checksum"] [take=6.385818026s] ``` -## Working principles +## Implementation principles -BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read in the restoration. +BR sends the backup and restoration commands to each TiKV node. After receiving these commands, TiKV performs the corresponding backup and restoration operations. Each TiKV node has a path in which the backup files generated in the backup operation are stored and from which the stored backup files are read during the restoration. -### Backup +### Backup principle When BR performs a backup operation, it first obtains the following information from PD: @@ -327,9 +357,9 @@ When BR performs a backup operation, it first obtains the following information According to these information, BR starts a TiDB instance internally to obtain the database or table information corresponding to the TS, and filters out the system databases (`information_schema`, `performance_schema`, `mysql`) at the same time. -According to the backup sub-command, two types of backup logic are available: +According to the backup sub-command, BR adopts the following two types of backup logic: -- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to every table. +- Full backup: BR traverses all the tables and constructs the KV range to be backed up according to each table. - Single table backup: BR constructs the KV range to be backed up according a single table. Finally, BR collects the KV range to be backed up and sends the complete backup request to the TiKV node of the cluster. @@ -338,39 +368,39 @@ The structure of the request: ``` backup.BackupRequest{ - ClusterId: clusterID, // The cluster ID + ClusterId: clusterID, // The cluster ID. StartKey: startKey, // The starting key of the backup (backed up) EndKey: endKey, // The ending key of the backup (not backed up) StartVersion: backupTS, // The backup snapshot time ... - Path: path, // The path in which backup files are stored + Path: path, // The path where backup files are stored RateLimit: rateLimit, // Backup speed (MB/s) Concurrency: concurrency, // The number of threads for the backup operation (4 by default) } ``` -After receiving the backup request, the TiKV node traverses all Region Leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backups some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. +After receiving the backup request, the TiKV node traverses all Region leaders on the node to find the Regions that overlap with the KV ranges in this request. The TiKV node backs up some or all of the data within the range, and generates the corresponding SST file (named in the format of `storeID_regionID_regionEpoch_tableID`) in the backup path. -After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the backupMeta file which is used for restoration. +After finishing backing up the data of the corresponding Region, the TiKV node returns the metadata to BR. BR collects the metadata and stores it in the `backupMeta` file which is used for restoration. If checksum is enabled when you execute the backup command, BR calculates the checksum of each backed up table for data check. -### Restoration +### Restoration principle -When BR performs the restoration, BR performs the following tasks in order: +During the data restoration process, BR performs the following tasks in order: -1. It parses the backupMeta file in the backup path, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. +1. It parses the `backupMeta` file in the backup path, and then starts a TiDB instance internally to create the corresponding databases and tables based on the parsed information. -2. It aggregates the parsed SST file according to the tables and `GroupBy`. +2. It aggregates the parsed SST files according to the tables and `GroupBy`. 3. It pre-splits Regions according to the key range of the SST file so that every Region corresponds to at least one SST file. -4. It traverses every table to be restored and the SST file corresponding to these tables. +4. It traverses each table to be restored and the SST file corresponding to each tables. 5. It finds the Region corresponding to the SST file and sends a request to the corresponding TiKV node for downloading the file. Then it sends a request for loading the file after the file is successfully downloaded. After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file is deleted asynchronously. -After the restoration operation, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. +After the restoration operation is complete, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. ![br-arch](/media/br-arch.png) diff --git a/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md b/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md index a018dac40591c..a34a662c7078b 100644 --- a/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md +++ b/v3.1/how-to/maintain/backup-and-restore/mydumper-loader.md @@ -1,13 +1,13 @@ --- -title: Use `mydumper`/`loader` to Backup and Restore -summary: Learn how to backup and restore the data of TiDB using `mydumper`/`loader`. +title: Use `mydumper`and `loader` to Back up and Restore Data +summary: Learn how to back up and restore the data of TiDB using `mydumper` and `loader`. category: how-to aliases: ['/docs/v3.1/how-to/maintain/backup-and-restore/'] --- -# Use `mydumper`/`loader` to Backup and Restore +# Use `mydumper` and `loader` to Back up and Restore Data -This document describes how to back up and restore the data of TiDB using `mydumper`/`loader`. Currently, this document only covers full backup and restoration. +This document describes how to back up and restore the data of TiDB using `mydumper` and `loader`. Currently, this document only covers full backup and restoration. Suppose that the TiDB service information is as follows: From 0d962a276de83b0c1bdc5c7fb6b444c754484993 Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 6 Dec 2019 17:41:25 +0800 Subject: [PATCH 18/20] maintain: fix a typo --- dev/how-to/maintain/backup-and-restore/br.md | 12 ++++++------ v3.1/how-to/maintain/backup-and-restore/br.md | 12 ++++++------ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 35b01eced83c3..4e7b70e49386d 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -369,13 +369,13 @@ The structure of the request: ``` backup.BackupRequest{ ClusterId: clusterID, // The cluster ID. - StartKey: startKey, // The starting key of the backup (backed up) - EndKey: endKey, // The ending key of the backup (not backed up) - StartVersion: backupTS, // The backup snapshot time + StartKey: startKey, // The starting key of the backup (backed up). + EndKey: endKey, // The ending key of the backup (not backed up). + StartVersion: backupTS, // The backup snapshot time. ... - Path: path, // The path where backup files are stored - RateLimit: rateLimit, // Backup speed (MB/s) - Concurrency: concurrency, // The number of threads for the backup operation (4 by default) + Path: path, // The path where backup files are stored. + RateLimit: rateLimit, // Backup speed (MB/s). + Concurrency: concurrency, // The number of threads for the backup operation (4 by default). } ``` diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index ea85566e92f7b..6c2c2482a8cac 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -369,13 +369,13 @@ The structure of the request: ``` backup.BackupRequest{ ClusterId: clusterID, // The cluster ID. - StartKey: startKey, // The starting key of the backup (backed up) - EndKey: endKey, // The ending key of the backup (not backed up) - StartVersion: backupTS, // The backup snapshot time + StartKey: startKey, // The starting key of the backup (backed up). + EndKey: endKey, // The ending key of the backup (not backed up). + StartVersion: backupTS, // The backup snapshot time. ... - Path: path, // The path where backup files are stored - RateLimit: rateLimit, // Backup speed (MB/s) - Concurrency: concurrency, // The number of threads for the backup operation (4 by default) + Path: path, // The path where backup files are stored. + RateLimit: rateLimit, // Backup speed (MB/s). + Concurrency: concurrency, // The number of threads for the backup operation (4 by default). } ``` From ce9bb44e7dd6b38b227281e8419c2ce3bccc5b7c Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 6 Dec 2019 17:45:12 +0800 Subject: [PATCH 19/20] maintain: fix grammar error --- dev/how-to/maintain/backup-and-restore/br.md | 2 +- v3.1/how-to/maintain/backup-and-restore/br.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 4e7b70e49386d..7265c1ccb6498 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -401,6 +401,6 @@ During the data restoration process, BR performs the following tasks in order: After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file is deleted asynchronously. -After the restoration operation is complete, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. +After the restoration operation is completed, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. ![br-arch](/media/br-arch.png) diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 6c2c2482a8cac..325c6e6c466fb 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -401,6 +401,6 @@ During the data restoration process, BR performs the following tasks in order: After TiKV receives the request to load the SST file, TiKV uses the Raft mechanism to ensure the strong consistency of the SST data. After the downloaded SST file is loaded successfully, the file is deleted asynchronously. -After the restoration operation is complete, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. +After the restoration operation is completed, BR performs a checksum calculation on the restored data to compare the stored data with the backed up data. ![br-arch](/media/br-arch.png) From d9ae454b3db60097caf0016c1e162ebaaee400cd Mon Sep 17 00:00:00 2001 From: TomShawn <1135243111@qq.com> Date: Fri, 6 Dec 2019 17:57:31 +0800 Subject: [PATCH 20/20] maintain: fix a typo --- dev/how-to/maintain/backup-and-restore/br.md | 2 +- v3.1/how-to/maintain/backup-and-restore/br.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/dev/how-to/maintain/backup-and-restore/br.md b/dev/how-to/maintain/backup-and-restore/br.md index 7265c1ccb6498..ad17ccabcf8fd 100644 --- a/dev/how-to/maintain/backup-and-restore/br.md +++ b/dev/how-to/maintain/backup-and-restore/br.md @@ -256,7 +256,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Examples -This section shows how to back up and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. +This section shows how to back up and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data volume. ### Data volume and machine configuration diff --git a/v3.1/how-to/maintain/backup-and-restore/br.md b/v3.1/how-to/maintain/backup-and-restore/br.md index 325c6e6c466fb..2b108d429cd49 100644 --- a/v3.1/how-to/maintain/backup-and-restore/br.md +++ b/v3.1/how-to/maintain/backup-and-restore/br.md @@ -256,7 +256,7 @@ In the above command, `--table` specifies the name of the table to be restored. ## Examples -This section shows how to back up and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data size. +This section shows how to back up and restore the data of an existing cluster. You can estimate the performance of backup and restoration based on machine performance, configuration and data volume. ### Data volume and machine configuration