sql-statements: document BACKUP, RESTORE and SHOW [BACKUPS|RESTORES]

TOC.md

@@ -87,6 +87,7 @@
       - [Use Mydumper and TiDB Lightning](/backup-and-restore-using-mydumper-lightning.md)
       - [Use BR](/br/backup-and-restore-tool.md)
       - [BR Usage Scenarios](/br/backup-and-restore-use-cases.md)
+      - [BR storages](/br/backup-and-restore-storages.md)
     + Identify Abnormal Queries
       - [Identify Slow Queries](/identify-slow-queries.md)
       - [Identify Expensive Queries](/identify-expensive-queries.md)
@@ -175,6 +176,7 @@
       - [`ALTER TABLE`](/sql-statements/sql-statement-alter-table.md)
       - [`ALTER USER`](/sql-statements/sql-statement-alter-user.md)
       - [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md)
+      - [`BACKUP`](/sql-statements/sql-statement-backup.md)
       - [`BEGIN`](/sql-statements/sql-statement-begin.md)
       - [`COMMIT`](/sql-statements/sql-statement-commit.md)
       - [`CREATE DATABASE`](/sql-statements/sql-statement-create-database.md)
@@ -213,13 +215,15 @@
       - [`RENAME INDEX`](/sql-statements/sql-statement-rename-index.md)
       - [`RENAME TABLE`](/sql-statements/sql-statement-rename-table.md)
       - [`REPLACE`](/sql-statements/sql-statement-replace.md)
+      - [`RESTORE`](/sql-statements/sql-statement-restore.md)
       - [`REVOKE <privileges>`](/sql-statements/sql-statement-revoke-privileges.md)
       - [`ROLLBACK`](/sql-statements/sql-statement-rollback.md)
       - [`SELECT`](/sql-statements/sql-statement-select.md)
       - [`SET [NAMES|CHARACTER SET]`](/sql-statements/sql-statement-set-names.md)
       - [`SET PASSWORD`](/sql-statements/sql-statement-set-password.md)
       - [`SET TRANSACTION`](/sql-statements/sql-statement-set-transaction.md)
       - [`SET [GLOBAL|SESSION] <variable>`](/sql-statements/sql-statement-set-variable.md)
+      - [`SHOW [BACKUPS|RESTORES]`](/sql-statements/sql-statement-show-backups.md)
       - [`SHOW CHARACTER SET`](/sql-statements/sql-statement-show-character-set.md)
       - [`SHOW COLLATION`](/sql-statements/sql-statement-show-collation.md)
       - [`SHOW [FULL] COLUMNS FROM`](/sql-statements/sql-statement-show-columns-from.md)

br/backup-and-restore-storages.md

@@ -0,0 +1,82 @@
+---
+title: BR storages
+summary: Describes the storage URL format used in BR.
+category: reference
+---
+
+# BR storages
+
+BR supports reading and writing data on the local filesystem, as well as on Amazon S3 and Google Cloud Storage. These are distinguished by the URL scheme in the `--storage` parameter passed into BR.
+
+## Schemes
+
+The following services are supported:
+
+| Service | Schemes | Example URL |
+|---------|---------|-------------|
+| Local filesystem, distributed on every node | local | `local:///path/to/dest/` |
+| Amazon S3 and compatible services | s3 | `s3://bucket-name/prefix/of/dest/` |
+| Google Cloud Storage (GCS) | gcs, gs | `gcs://bucket-name/prefix/of/dest/` |
+| Write to nowhere (for benchmarking only) | noop | `noop://` |
+
+## Parameters
+
+Cloud storages such as S3 and GCS sometimes require additional configuration for connection. You can specify parameters for such configuration. For example:
+
+{{< copyable "shell-regular" >}}
+
+```shell
+./br backup full -u 127.0.0.1:2379 -s 's3://bucket-name/prefix?region=us-west-2'
+```
+
+### S3 parameters
+
+| Parameter | Description |
+|----------:|---------|
+| `access-key` | The access key |
+| `secret-access-key` | The secret access key |
+| `region` | Service Region for Amazon S3 (default to `us-east-1`) |
+| `use-accelerate-endpoint` | Whether to use the accelerate endpoint on Amazon S3 (default to `false`) |
+| `endpoint` | URL of custom endpoint for S3-compatible services (for example, `https://s3.example.com/`) |
+| `force-path-style` | Use path style access rather than virtual hosted style access (default to `false`) |
+| `storage-class` | Storage class of the uploaded objects (for example, `STANDARD`, `STANDARD_IA`) |
+| `sse` | Server-side encryption algorithm used to encrypt the upload (empty, `AES256` or `aws:kms`) |
+| `sse-kms-key-id` | If `sse` is set to `aws:kms`, specifies the KMS ID |
+| `acl` | Canned ACL of the uploaded objects (for example, `private`, `authenticated-read`) |
+
+> **Note:**
+>
+> It is not recommended to pass in the access key and secret access key directly in the storage URL, because these keys are logged in plain text. BR tries to infer these keys from the environment in the following order:
+
+1. `$AWS_ACCESS_KEY_ID` and `$AWS_SECRET_ACCESS_KEY` environment variables
+2. `$AWS_ACCESS_KEY` and `$AWS_SECRET_KEY` environment variables
+3. Shared credentials file on the BR node at the path specified by the `$AWS_SHARED_CREDENTIALS_FILE` environment variable
+4. Shared credentials file on the BR node at `~/.aws/credentials`
+5. Current IAM role of the Amazon EC2 container
+6. Current IAM role of the Amazon ECS task
+
+### GCS parameters
+
+| Parameter | Description |
+|----------:|---------|
+| `credentials-file` | The path to the credentials JSON file on the TiDB node |
+| `storage-class` | Storage class of the uploaded objects (for example, `STANDARD`, `COLDLINE`) |
+| `predefined-acl` | Predefined ACL of the uploaded objects (for example, `private`, `project-private`) |
+
+When `credentials-file` is not specified, BR will try to infer the credentials from the environment, in the following order:
+
+1. Content of the file on the BR node at the path specified by the `$GOOGLE_APPLICATION_CREDENTIALS` environment variable
+2. Content of the file on the BR node at `~/.config/gcloud/application_default_credentials.json`
+3. When running in GCE or GAE, the credentials fetched from the metadata server.
+
+## Sending credentials to TiKV
+
+By default, when using S3 and GCS destinations, BR will send the credentials to every TiKV nodes to reduce setup complexity.
+
+However, this is unsuitable on cloud environment, where every node has their own role and permission. In such cases, you need to disable credentials sending with `--send-credentials-to-tikv=false` (or the short form `-c=0`):
+
+{{< copyable "shell-regular" >}}
+
+```shell
+./br backup full -c=0 -u pd-service:2379 -s 's3://bucket-name/prefix'
+```

br/backup-and-restore-tool.md

@@ -263,7 +263,7 @@ To restore the cluster data, use the `br restore` command. You can add the `full
 > - Data are replicated into multiple peers. When ingesting SSTs, these files have to be present on *all* peers. This is unlike back up where reading from a single node is enough.
 > - Where each peer is scattered to during restore is random. We don't know in advance which node will read which file.
 >
-> These can be avoided using shared storage, e.g. mounting an NFS on the local path, or using S3. With network storage, every node can automatically read every SST file, so these caveats no longer apply.
+> These can be avoided using shared storage, for example mounting an NFS on the local path, or using S3. With network storage, every node can automatically read every SST file, so these caveats no longer apply.
 
 ### Restore all the backup data
 

sql-statements/sql-statement-backup.md

@@ -0,0 +1,190 @@
+---
+title: BACKUP | TiDB SQL Statement Reference
+summary: An overview of the usage of BACKUP for the TiDB database.
+category: reference
+---
+
+# BACKUP
+
+This statement is used to perform a distributed backup of the TiDB cluster.
+
+The `BACKUP` statement uses the same engine as the [BR tool](/br/backup-and-restore-use-cases.md) does, except that the backup process is driven by TiDB itself rather than a separate BR tool. All benefits and warnings of BR also apply in this statement.
+
+Executing `BACKUP` requires `SUPER` privilege. Additionally, both the TiDB node executing the backup and all TiKV nodes in the cluster must have read or write permission to the destination.
+
+The `BACKUP` statement is blocked until the entire backup task is finished, failed, or canceled. A long-lasting connection should be prepared for executing `BACKUP`. The task can be canceled using the [`KILL TIDB QUERY`](/sql-statements/sql-statement-kill.md) statement.
+
+Only one `BACKUP` and [`RESTORE`](/sql-statements/sql-statement-restore.md) task can be executed at a time. If a `BACKUP` or `RESTORE` statement is already being executed on the same TiDB server, the new `BACKUP` execution will wait until all previous tasks are finished.
+
+`BACKUP` can only be used with "tikv" storage engine. Using `BACKUP` with the "mocktikv" engine will fail.
+
+## Synopsis
+
+**BackupStmt:**
+
+![BackupStmt](/media/sqlgram/BackupStmt.png)
+
+**BRIETables:**
+
+![BRIETables](/media/sqlgram/BRIETables.png)
+
+**BackupOption:**
+
+![BackupOption](/media/sqlgram/BackupOption.png)
+
+**Boolean:**
+
+![Boolean](/media/sqlgram/Boolean.png)
+
+**BackupTSO:**
+
+![BackupTSO](/media/sqlgram/BackupTSO.png)
+
+## Examples
+
+### Back up databases
+
+{{< copyable "sql" >}}
+
+```sql
+BACKUP DATABASE `test` TO 'local:///mnt/backup/2020/04/';
+```
+
+```sql
++------------------------------+-----------+-----------------+---------------------+---------------------+
+| Destination                  | Size      | BackupTS        | Queue Time          | Execution Time      |
++------------------------------+-----------+-----------------+---------------------+---------------------+
+| local:///mnt/backup/2020/04/ | 248665063 | 416099531454472 | 2020-04-12 23:09:48 | 2020-04-12 23:09:48 |
++------------------------------+-----------+-----------------+---------------------+---------------------+
+1 row in set (58.453 sec)
+```
+
+In the example above, the `test` database is backed up into the local filesystem. The data is saved as SST files in the `/mnt/backup/2020/04/` directories distributed among all TiDB and TiKV nodes.
+
+The first row of the result above is described as follows:
+
+| Column | Description |
+| :-------- | :--------- |
+| `Destination` | The destination URL |
+| `Size` |  The total size of the backup archive, in bytes |
+| `BackupTS` | The TSO of the snapshot when the backup is created (useful for [incremental backup](#incremental-backup)) |
+| `Queue Time` | The timestamp (in current time zone) when the `BACKUP` task is queued. |
+| `Execution Time` | The timestamp (in current time zone) when the `BACKUP` task starts to run. |
+
+### Back up tables
+
+{{< copyable "sql" >}}
+
+```sql
+BACKUP TABLE `test`.`sbtest01` TO 'local:///mnt/backup/sbtest01/';
+```
+
+{{< copyable "sql" >}}
+
+```sql
+BACKUP TABLE sbtest02, sbtest03, sbtest04 TO 'local:///mnt/backup/sbtest/';
+```
+
+### Back up the entire cluster
+
+{{< copyable "sql" >}}
+
+```sql
+BACKUP DATABASE * TO 'local:///mnt/backup/full/';
+```
+
+Note that the system tables (`mysql.*`, `INFORMATION_SCHEMA.*`, `PERFORMANCE_SCHEMA.*`, …) will not be included into the backup.
+
+### Remote destinations
+
+BR supports backing up data to S3 or GCS:
+
+{{< copyable "sql" >}}
+
+```sql
+BACKUP DATABASE `test` TO 's3://example-bucket-2020/backup-05/?region=us-west-2';
+```
+
+The URL syntax is further explained in [BR storages](/br/backup-and-restore-storages.md).
+
+When running on cloud environment where credentials should not be distributed, set the `SEND_CREDENTIALS_TO_TIKV` option to `FALSE`:
+
+{{< copyable "sql" >}}
+
+```sql
+BACKUP DATABASE `test` TO 's3://example-bucket-2020/backup-05/?region=us-west-2'
+    SEND_CREDENTIALS_TO_TIKV = FALSE;
+```
+
+### Performance fine-tuning
+
+Use `RATE_LIMIT` to limit the average upload speed per TiKV node to reduce network bandwidth.
+
+By default, every TiKV node would run 4 backup threads. This value can be adjusted with the `CONCURRENCY` option.
+
+Before backup is completed, `BACKUP` would perform a checksum against the data on the cluster to verify correctness. This step can be disabled with the `CHECKSUM` option if you are confident that this is unnecessary.
+
+{{< copyable "sql" >}}
+
+```sql
+BACKUP DATABASE `test` TO 's3://example-bucket-2020/backup-06/'
+    RATE_LIMIT = 120 MB/SECOND
+    CONCURRENCY = 8
+    CHECKSUM = FALSE;
+```
+
+### Snapshot
+
+Specify a timestamp, TSO or relative time to backup historical data.
+
+{{< copyable "sql" >}}
+
+```sql
+-- relative time
+BACKUP DATABASE `test` TO 'local:///mnt/backup/hist01'
+    SNAPSHOT = 36 HOUR AGO;
+
+-- timestamp (in current time zone)
+BACKUP DATABASE `test` TO 'local:///mnt/backup/hist02'
+    SNAPSHOT = '2020-04-01 12:00:00';
+
+-- timestamp oracle
+BACKUP DATABASE `test` TO 'local:///mnt/backup/hist03'
+    SNAPSHOT = 415685305958400;
+```
+
+The supported units for relative time are:
+
+* MICROSECOND
+* SECOND
+* MINUTE
+* HOUR
+* DAY
+* WEEK
+
+Note that, following SQL standard, the units are always singular.
+
+### Incremental backup
+
+Supply the `LAST_BACKUP` option to only backup the changes between the last backup to the current snapshot.
+
+{{< copyable "sql" >}}
+
+```sql
+-- timestamp (in current time zone)
+BACKUP DATABASE `test` TO 'local:///mnt/backup/hist02'
+    LAST_BACKUP = '2020-04-01 12:00:00';
+
+-- timestamp oracle
+BACKUP DATABASE `test` TO 'local:///mnt/backup/hist03'
+    LAST_BACKUP = 415685305958400;
+```
+
+## MySQL compatibility
+
+This statement is a TiDB extension to MySQL syntax.
+
+## See also
+
+* [RESTORE](/sql-statements/sql-statement-restore.md)
+* [SHOW BACKUPS](/sql-statements/sql-statement-show-backups.md)