Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update tikv docs for titan fallback #10257

Merged
merged 6 commits into from Sep 14, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 1 addition & 2 deletions experimental-features.md
Expand Up @@ -38,7 +38,6 @@ Elastic scheduling feature. It enables the TiDB cluster to dynamically scale out

## Storage

+ [Disable Titan](/storage-engine/titan-configuration.md#disable-titan-experimental) (Introduced in v4.0)
+ [Titan Level Merge](/storage-engine/titan-configuration.md#level-merge-experimental) (Introduced in v4.0)
+ Divide Regions are divided into buckets. [Buckets are used as the unit of concurrent query](/tune-region-performance.md#use-bucket-to-increase-concurrency) to improve the scan concurrency. (Introduced in v6.1.0)
+ TiKV introduces [API V2](/tikv-configuration-file.md#api-version-new-in-v610). (Introduced in v6.1.0)
Expand All @@ -49,7 +48,7 @@ Elastic scheduling feature. It enables the TiDB cluster to dynamically scale out
+ [Configure disk quota](/tidb-lightning/tidb-lightning-physical-import-mode-usage.md#configure-disk-quota-new-in-v620) for TiDB Lightning. (Introduced in v6.2.0)
+ [Continuous Data Validation in DM](/dm/dm-continuous-data-validation.md) (Introduced in v6.2.0)

## 数据共享订阅
## Data share subscription

+ [Cross-cluster RawKV replication](/tikv-configuration-file.md#api-version-new-in-v610) (Introduced in v6.2.0)

Expand Down
31 changes: 25 additions & 6 deletions storage-engine/titan-configuration.md
Expand Up @@ -45,7 +45,7 @@ If you want to speed up the writing process, compact data of the whole TiKV clus

> **Note:**
>
> When Titan is disabled, RocksDB cannot read data that has been migrated to Titan. If Titan is incorrectly disabled on a TiKV instance with Titan already enabled (mistakenly set `rocksdb.titan.enabled` to `false`), TiKV will fail to start, and the `You have disabled titan when its data directory is not empty` error appears in the TiKV log. To correctly disabled Titan, see [Disable Titan](#disable-titan-experimental).
> When Titan is disabled, RocksDB cannot read data that has been migrated to Titan. If Titan is incorrectly disabled on a TiKV instance with Titan already enabled (mistakenly set `rocksdb.titan.enabled` to `false`), TiKV will fail to start, and the `You have disabled titan when its data directory is not empty` error appears in the TiKV log. To correctly disabled Titan, see [Disable Titan](#disable-titan).

## Parameters

Expand Down Expand Up @@ -110,19 +110,38 @@ To adjust Titan-related parameters using TiUP, refer to [Modify the configuratio
rate-bytes-per-sec = 0
```

## Disable Titan (experimental)
## Disable Titan

To disable Titan, you can configure the `rocksdb.defaultcf.titan.blob-run-mode` option. The optional values for `blob-run-mode` are as follows:

- When the option is set to `normal`, Titan performs read and write operations normally.
- When the option is set to `read-only`, all newly written values are written into RocksDB, regardless of the value size.
- When the option is set to `fallback`, all newly written values are written into RocksDB, regardless of the value size. Also, all compacted values stored in the Titan blob file are automatically moved back to RocksDB.

To disable Titan, set `blob-run-mode = "fallback"` and perform a full compaction using tikv-ctl. After that, check the monitoring metrics, confirm that the blob file size decreases to `0`. Then you can set `rocksdb.titan.enabled` to `false` and restart TiKV.
To fully disable Titan for all existing and future data, you can follow these steps:

> **Warning:**
>
> Disabling Titan is an experimental feature. It is **NOT** recommended to use it if not necessary.
+ First update the configuration of the TiKV nodes you wish to disable Titan. This can be done either with `tiup cluster edit-config` followed by `tiup cluster reload`, or manually updating the configuration file and restarting TiKV.
tabokie marked this conversation as resolved.
Show resolved Hide resolved

```toml
[rocksdb.defaultcf.titan]
blob-run-mode = "fallback"
discardable-ratio = 1.0
```

+ Then perform a full compaction using tikv-ctl. This process will consume large amount of I/O and CPU resources.
tabokie marked this conversation as resolved.
Show resolved Hide resolved

```bash
tikv-ctl --pd <PD_ADDR> compact-cluster --bottommost force
```

+ After the compaction is finished, you should wait for the "Blob file count" metrics under "TiKV-Details/Titan - kv" to decrease to `0`.
Copy link
Contributor

@ran-huang ran-huang Sep 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this document, the name of the monitoring panel is "Titan - All", rather than "Titan - kv". Can you confirm which is correct? @tabokie

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's dynamically named. When titan is enabled, it will be "Titan - kv".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Thanks.


+ Finally, update the configuration of these TiKV nodes to disable Titan.
tabokie marked this conversation as resolved.
Show resolved Hide resolved

```toml
[rocksdb.titan]
enabled = false
```

## Level Merge (experimental)

Expand Down
4 changes: 2 additions & 2 deletions tikv-configuration-file.md
Expand Up @@ -482,8 +482,8 @@ Configuration items related to the I/O rate limiter.
### `mode`

+ Determines which types of I/O operations are counted and restrained below the `max-bytes-per-sec` threshold. Currently, only the write-only mode is supported.
+ Optional value: `"write-only"`
+ Default value: `"write-only"`
+ Optional values: `"read-only"`, `"write-only"`, `"all-io"`
ran-huang marked this conversation as resolved.
Show resolved Hide resolved
+ Default value: `"all-io"`
tabokie marked this conversation as resolved.
Show resolved Hide resolved

## raftstore

Expand Down