Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TiKV API V2 docs #8719

Merged
merged 11 commits into from
Jun 7, 2022
36 changes: 36 additions & 0 deletions tikv-configuration-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,23 @@ Configuration items related to the sharing of block cache among multiple RocksDB
+ Default value: 45% of the size of total system memory
+ Unit: KB|MB|GB

### `api-version` <span class="version-mark">New in v6.1.0</span>

TomShawn marked this conversation as resolved.
Show resolved Hide resolved
+ The storage format and interface version used by TiKV when TiKV serves as the raw key-value store.
+ Value options:
+ `1`: Uses API V1, does not encode the data passed from the client, and stores data as it is. In versions earlier than v6.1.0, TiKV uses API V1 by default.
+ `2`: Uses API V2:
+ The data is stored in the MVCC (Multi Version Concurrency Control) format, where the timestamp is obtained from PD (which is TSO) by tikv-server.
TomShawn marked this conversation as resolved.
Show resolved Hide resolved
Oreoxmt marked this conversation as resolved.
Show resolved Hide resolved
+ When API V2 is used, you are expected to set `storage.enable-ttl = true` at the same time. Because API V2 supports the TTL feature, you must turn on `enable-ttl` explicitly. Otherwise, it will be in conflict because `storage.enable-ttl` defaults to `false`.
+ When API V2 is enabled, you need to deploy at least one tidb-server instance to reclaim expired data. Note that this tidb-server instance cannot provide read or write services. To ensure high availability, you can deploy multiple tidb-server instances.
+ Client support is required for API V2. For details, see the corresponding instruction of the client for the API V2.
+ Default value: `1`

> **Warning:**
>
> - You can set the value of `api-version` to `2` **only when** deploying a new TiKV cluster. **Do not** modify the value of this configuration item in an existing TiKV cluster. TiKV clusters with different `api-version` values use different data formats. Therefore, if you modify the value of this item in an existing TiKV cluster, the cluster will store data in different formats and causes data corruption. It will raise the "unable to switch storage.api_version" error when you start the TiKV cluster.
> - After API V2 is enabled, you **cannot** downgrade the TiKV cluster to a version earlier than v6.1.0. Otherwise, data corruption might occur.

## storage.flow-control

Configuration items related to the flow control mechanism in TiKV. This mechanism replaces the write stall mechanism in RocksDB and controls flow at the scheduler layer, which avoids secondary disasters caused by the stuck Raftstore or Apply threads.
Expand Down Expand Up @@ -1590,3 +1607,22 @@ Suppose that your machine on which TiKV is deployed has limited resources, for e

+ The maximum time that a single read or write request is forced to wait before it is processed in the foreground.
+ Default value: `500ms`

## causal-ts <span class="version-mark">New in v6.1.0</span>

TomShawn marked this conversation as resolved.
Show resolved Hide resolved
Configuration items related to getting the timestamp when TiKV API V2 is enabled (`storage.api-version = 2`).

To reduce write latency and avoid frequent access to PD, TiKV periodically fetches and caches a batch of timestamps in the local. When the locally cached timestamps are exhausted, TiKV immediately makes a timestamp request. In this situation, the latency of some write requests are increased. To reduce the occurrence of this situation, TiKV dynamically adjusts the size of the locally cached timestamps according to the workload. For most of the time, you do not need to adjust the following parameters.

### `renew-interval`

+ The interval at which the locally cached timestamps are refreshed.
+ At an interval of `renew-interval`, TiKV starts a batch of timestamp refresh and adjusts the number of cached timestamps according to the timestamp consumption in the previous period. If you set this parameter to too large a value, the latest TiKV workload changes are not reflected in time. If you set this parameter to too small a value, the load of PD increases. If the write traffic is strongly fluctuating, if timestamps are frequently exhausted, and if write latency increases, you can set this parameter to a smaller value. At the same time, you should also consider the load of PD.
+ Default value: `"100ms"`

### `renew-batch-min-size`

+ The minimum number of locally cached timestamps.
+ TiKV adjusts the number of cached timestamps according to the timestamp consumption in the previous period. If the usage of locally cached timestamps is low, TiKV gradually reduces the number of cached timestamps until it reaches `renew-batch-min-size`. If large bursty write traffic often occurs in your application, you can set this parameter to a larger value as appropriate. Note that this parameter is the cache size for a single tikv-server. If you set the parameter to too large a value and the cluster contains many tikv-servers, the TSO consumption will be too fast.
+ In the **TiKV-RAW** \> **Causal timestamp** panel in Grafana, **TSO batch size** is the number of locally cached timestamps that has been dynamically adjusted according to the application workload. You can refer to this metric to adjust `renew-batch-min-size`.
+ Default value: `100`