From 691835464b349132355f72300df4330eee579be2 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Sat, 16 Mar 2024 13:51:10 +0800 Subject: [PATCH 1/8] Add temp.md --- temp.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 temp.md diff --git a/temp.md b/temp.md new file mode 100644 index 0000000000000..af27ff4986a7b --- /dev/null +++ b/temp.md @@ -0,0 +1 @@ +This is a test file. \ No newline at end of file From ecd175c2ade164e39c3fa75f73005121b4c8ff2a Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Sat, 16 Mar 2024 13:51:14 +0800 Subject: [PATCH 2/8] Delete temp.md --- temp.md | 1 - 1 file changed, 1 deletion(-) delete mode 100644 temp.md diff --git a/temp.md b/temp.md deleted file mode 100644 index af27ff4986a7b..0000000000000 --- a/temp.md +++ /dev/null @@ -1 +0,0 @@ -This is a test file. \ No newline at end of file From 26d268d705118b332f24a3b788889273c487d285 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Sat, 16 Mar 2024 14:29:45 +0800 Subject: [PATCH 3/8] add translation --- dynamic-config.md | 3 +++ storage-engine/titan-configuration.md | 6 +++++- tikv-configuration-file.md | 5 +++++ 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/dynamic-config.md b/dynamic-config.md index d006a759ae8e6..dd9273ae767b5 100644 --- a/dynamic-config.md +++ b/dynamic-config.md @@ -213,6 +213,9 @@ The following TiKV configuration items can be modified dynamically: | `{db-name}.{cf-name}.soft-pending-compaction-bytes-limit` | The soft limit on the pending compaction bytes | | `{db-name}.{cf-name}.hard-pending-compaction-bytes-limit` | The hard limit on the pending compaction bytes | | `{db-name}.{cf-name}.titan.blob-run-mode` | The mode of processing blob files | +| `{db-name}.{cf-name}.titan.min-blob-size` | The threshold at which data is stored in Titan. When the value of the data reaches this threshold, it will be stored in a Titan blob file. | +| `{db-name}.{cf-name}.titan.blob-file-compression` | The compression algorithm used by Titan blob files | +| `{db-name}.{cf-name}.titan.discardable-ratio` | The threshold of garbage data ratio in Titan data files for GC. When the ratio of useless data in a blob file exceeds the threshold, Titan GC is triggered. | | `server.grpc-memory-pool-quota` | Limits the memory size that can be used by gRPC | | `server.max-grpc-send-msg-len` | Sets the maximum length of a gRPC message that can be sent | | `server.snap-io-max-bytes-per-sec` | Sets the maximum allowable disk bandwidth when processing snapshots | diff --git a/storage-engine/titan-configuration.md b/storage-engine/titan-configuration.md index 5f17f5f641b79..b313c4645dea7 100644 --- a/storage-engine/titan-configuration.md +++ b/storage-engine/titan-configuration.md @@ -105,6 +105,10 @@ If you observe that the Titan GC thread is in full load for a long time from **T You can adjust [`rate-bytes-per-sec`](/tikv-configuration-file.md#rate-bytes-per-sec) to limit the I/O rate of RocksDB compaction, reducing its impact on foreground read and write performance during high traffic. +### `shared-blob-cache` (New in v8.0.0) + +You can control whether to enable the shared cache for Titan blob files and RocksDB block files through [`shared-blob-cache`](/tikv-configuration-file.md#shared-blob-cache-tidb-new-in-v800). The default value is `true`. When the shared cache is enabled, block files have higher priority. This means that TiKV prioritizes meeting the cache needs of block files and then uses the remaining cache for blob files. + ### Titan configuration example The following is an example of the Titan configuration file. You can either [use TiUP to modify the configuration](/maintain-tidb-using-tiup.md#modify-the-configuration) or [configure a TiDB cluster on Kubernetes](https://docs.pingcap.com/tidb-in-kubernetes/stable/configure-a-tidb-cluster). @@ -150,7 +154,7 @@ To fully disable Titan for all existing and future data, you can follow these st > **Note:** > - > When there is insufficient disk space to accommodate both Titan and RocksDB data, it is recommended to use the default value of `0.5` for [`discardable-ratio`](/tikv-configuration-file.md#discardable-ratio). In general, the default value is recommended when available disk space is less than 50%. This is because when `discardable-ratio = 1.0`, the RocksDB data continues to increase. At the same time, the recycling of existing blob files in Titan requires all the data in that file to be converted to RocksDB, which is a slow process. However, if the disk size is large enough, setting `discardable-ratio = 1.0` can reduce the GC of the blob file itself during compaction, which saves bandwidth. + > When there is insufficient disk space to accommodate both Titan and RocksDB data, it is recommended to use the default value of `0.5` for [`discardable-ratio`](/tikv-configuration-file.md#discardable-ratio). In general, the default value is recommended when available disk space is less than 50%. This is because when `discardable-ratio = 1.0`, the RocksDB data continues to increase. At the same time, the recycling of existing blob files in Titan requires all the data in that file to be converted to RocksDB, which is a slow process. However, if the disk size is large enough, setting `discardable-ratio = 1.0` can reduce the GC of the blob file itself during compaction, which saves bandwidth. When disk space is insufficient, executing `tikv-ctl compact cluster` might result in the entire cluster running out of available space and thus unable to write data. 2. (Optional) Perform a full compaction using tikv-ctl. This process will consume a large amount of I/O and CPU resources. diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 85dd3e89bfa87..7c0fac79047ca 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1661,6 +1661,11 @@ Configuration items related to `rocksdb.defaultcf.titan`. + Recommended value: After database stabilization, it is recommended to set the RocksDB block cache (`storage.block-cache.capacity`) based on monitoring to maintain a block cache hit rate of at least 95%, and set `blob-cache-size` to `(total memory size) * 50% - (size of block cache)`. This is to ensure that the block cache is sufficiently large to cache the entire RocksDB, while maximizing the blob cache size. However, to prevent a significant drop in the block cache hit rate, do not set the blob cache size too large. + Unit: KB|MB|GB +### `shared-blob-cache` (New in v8.0.0) + ++ Controls whether to enable the shared cache for Titan blob files and RocksDB block files. ++ Default value: `true`. When the shared cache is enabled, block files have higher priority. This means that TiKV prioritizes meeting the cache needs of block files and then uses the remaining cache for blob files. + ### `min-gc-batch-size` + The minimum total size of Blob files required to perform GC for one time From 4ddab1de744ef46ec210909b8c9e5f216cb0f423 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 21 Mar 2024 09:11:17 +0800 Subject: [PATCH 4/8] sync from zh changes --- storage-engine/titan-configuration.md | 6 +++++- tikv-configuration-file.md | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/storage-engine/titan-configuration.md b/storage-engine/titan-configuration.md index b313c4645dea7..689b56e76f8f9 100644 --- a/storage-engine/titan-configuration.md +++ b/storage-engine/titan-configuration.md @@ -154,10 +154,14 @@ To fully disable Titan for all existing and future data, you can follow these st > **Note:** > - > When there is insufficient disk space to accommodate both Titan and RocksDB data, it is recommended to use the default value of `0.5` for [`discardable-ratio`](/tikv-configuration-file.md#discardable-ratio). In general, the default value is recommended when available disk space is less than 50%. This is because when `discardable-ratio = 1.0`, the RocksDB data continues to increase. At the same time, the recycling of existing blob files in Titan requires all the data in that file to be converted to RocksDB, which is a slow process. However, if the disk size is large enough, setting `discardable-ratio = 1.0` can reduce the GC of the blob file itself during compaction, which saves bandwidth. When disk space is insufficient, executing `tikv-ctl compact cluster` might result in the entire cluster running out of available space and thus unable to write data. + > When there is insufficient disk space to accommodate both Titan and RocksDB data, it is recommended to use the default value of `0.5` for [`discardable-ratio`](/tikv-configuration-file.md#discardable-ratio). In general, the default value is recommended when available disk space is less than 50%. This is because when `discardable-ratio = 1.0`, the RocksDB data continues to increase. At the same time, the recycling of existing blob files in Titan requires all the data in that file to be converted to RocksDB, which is a slow process. However, if the disk size is large enough, setting `discardable-ratio = 1.0` can reduce the GC of the blob file itself during compaction, which saves bandwidth. 2. (Optional) Perform a full compaction using tikv-ctl. This process will consume a large amount of I/O and CPU resources. + > **Warning:** + > + > When disk space is insufficient, executing the following command might result in the entire cluster running out of available space and thus unable to write data. + ```bash tikv-ctl --pd compact-cluster --bottommost force ``` diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 7c0fac79047ca..f153a7b8edb8b 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1658,7 +1658,7 @@ Configuration items related to `rocksdb.defaultcf.titan`. + The cache size of a Blob file + Default value: `"0GB"` + Minimum value: `0` -+ Recommended value: After database stabilization, it is recommended to set the RocksDB block cache (`storage.block-cache.capacity`) based on monitoring to maintain a block cache hit rate of at least 95%, and set `blob-cache-size` to `(total memory size) * 50% - (size of block cache)`. This is to ensure that the block cache is sufficiently large to cache the entire RocksDB, while maximizing the blob cache size. However, to prevent a significant drop in the block cache hit rate, do not set the blob cache size too large. ++ Recommended value: `0`. Starting from v8.0.0, TiKV introduces the `shared-blob-cache` configuration item and enables it by default, so there is no need to set `blob-cache-size` separately. The `blob-cache-size` configuration only takes effect when `shared-blob-cache` is set to `false`. + Unit: KB|MB|GB ### `shared-blob-cache` (New in v8.0.0) From 95925d15b6d9d8f446fb5afe97573b73485d4e59 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 21 Mar 2024 09:14:30 +0800 Subject: [PATCH 5/8] Apply suggestions from code review --- dynamic-config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dynamic-config.md b/dynamic-config.md index dd9273ae767b5..cff4e08482ead 100644 --- a/dynamic-config.md +++ b/dynamic-config.md @@ -213,7 +213,7 @@ The following TiKV configuration items can be modified dynamically: | `{db-name}.{cf-name}.soft-pending-compaction-bytes-limit` | The soft limit on the pending compaction bytes | | `{db-name}.{cf-name}.hard-pending-compaction-bytes-limit` | The hard limit on the pending compaction bytes | | `{db-name}.{cf-name}.titan.blob-run-mode` | The mode of processing blob files | -| `{db-name}.{cf-name}.titan.min-blob-size` | The threshold at which data is stored in Titan. When the value of the data reaches this threshold, it will be stored in a Titan blob file. | +| `{db-name}.{cf-name}.titan.min-blob-size` | The threshold at which data is stored in Titan. Data is stored in a Titan blob file when its value reaches this threshold. | | `{db-name}.{cf-name}.titan.blob-file-compression` | The compression algorithm used by Titan blob files | | `{db-name}.{cf-name}.titan.discardable-ratio` | The threshold of garbage data ratio in Titan data files for GC. When the ratio of useless data in a blob file exceeds the threshold, Titan GC is triggered. | | `server.grpc-memory-pool-quota` | Limits the memory size that can be used by gRPC | From 8978f678b0f79d42c90d0fc3f082fe31f885c1e4 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 21 Mar 2024 09:17:03 +0800 Subject: [PATCH 6/8] Update tikv-configuration-file.md --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index f153a7b8edb8b..3c9b386f6ef06 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1658,7 +1658,7 @@ Configuration items related to `rocksdb.defaultcf.titan`. + The cache size of a Blob file + Default value: `"0GB"` + Minimum value: `0` -+ Recommended value: `0`. Starting from v8.0.0, TiKV introduces the `shared-blob-cache` configuration item and enables it by default, so there is no need to set `blob-cache-size` separately. The `blob-cache-size` configuration only takes effect when `shared-blob-cache` is set to `false`. ++ Recommended value: `0`. Starting from v8.0.0, TiKV introduces the `shared-blob-cache` configuration item and enables it by default, so there is no need to set `blob-cache-size` separately. The configuration of `blob-cache-size` only takes effect when `shared-blob-cache` is set to `false`. + Unit: KB|MB|GB ### `shared-blob-cache` (New in v8.0.0) From 1869f2a3570d96d6a66f3afd0ad506b3085ed1a3 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 22 Mar 2024 14:12:10 +0800 Subject: [PATCH 7/8] Update titan-configuration.md --- storage-engine/titan-configuration.md | 1 - 1 file changed, 1 deletion(-) diff --git a/storage-engine/titan-configuration.md b/storage-engine/titan-configuration.md index 689b56e76f8f9..b11eacc51d634 100644 --- a/storage-engine/titan-configuration.md +++ b/storage-engine/titan-configuration.md @@ -125,7 +125,6 @@ max-background-gc = 1 min-blob-size = "32KB" blob-file-compression = "zstd" zstd-dict-size = "16KB" -blob-cache-size = "0GB" discardable-ratio = 0.5 blob-run-mode = "normal" level-merge = false From f9be7051875dedb716c8332a78b629957678879b Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 22 Mar 2024 14:20:59 +0800 Subject: [PATCH 8/8] fix broken links --- storage-engine/titan-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/storage-engine/titan-configuration.md b/storage-engine/titan-configuration.md index b11eacc51d634..5f5ca31268eb0 100644 --- a/storage-engine/titan-configuration.md +++ b/storage-engine/titan-configuration.md @@ -107,7 +107,7 @@ You can adjust [`rate-bytes-per-sec`](/tikv-configuration-file.md#rate-bytes-per ### `shared-blob-cache` (New in v8.0.0) -You can control whether to enable the shared cache for Titan blob files and RocksDB block files through [`shared-blob-cache`](/tikv-configuration-file.md#shared-blob-cache-tidb-new-in-v800). The default value is `true`. When the shared cache is enabled, block files have higher priority. This means that TiKV prioritizes meeting the cache needs of block files and then uses the remaining cache for blob files. +You can control whether to enable the shared cache for Titan blob files and RocksDB block files through [`shared-blob-cache`](/tikv-configuration-file.md#shared-blob-cache-new-in-v800). The default value is `true`. When the shared cache is enabled, block files have higher priority. This means that TiKV prioritizes meeting the cache needs of block files and then uses the remaining cache for blob files. ### Titan configuration example