Skip to content
1 change: 1 addition & 0 deletions TOC-tidb-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,7 @@
- [TiDB Limitations](/tidb-limitations.md)
- Benchmarks
- [TPC-C Performance Test Report](/tidb-cloud/v7.1.0-performance-benchmarking-with-tpcc.md)
- [Sysbench Performance Test Report](/tidb-cloud/v7.1.0-performance-benchmarking-with-sysbench.md)
- SQL
- [Explore SQL with TiDB](/basic-sql-operations.md)
- SQL Language Structure and Syntax
Expand Down
Binary file added media/tidb-cloud/v.7.1.0-oltp_read_write.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/v7.1.0-oltp_insert.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/v7.1.0-oltp_select_point.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added media/tidb-cloud/v7.1.0-oltp_update_index.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
175 changes: 175 additions & 0 deletions tidb-cloud/v7.1.0-performance-benchmarking-with-sysbench.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
---
title: TiDB Cloud Sysbench Performance Test Report
summary: Introduce the Sysbench performance test results for TiDB Cloud.
---

# TiDB Cloud Sysbench Performance Test Report

This document provides the Sysbench performance test steps and results for a TiDB Dedicated cluster with the TiDB version of v7.1.0. This report can also be used as a reference for the performance of TiDB v7.1.0 clusters.

## Test overview

This test aims at showing the Sysbench performance of TiDB v7.1.0 in the Online Transactional Processing (OLTP) scenario.

## Test environment

### TiDB cluster

The test is conducted on a TiDB cluster with the following settings:

- Cluster type: [TiDB Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-dedicated)
- Cluster version: v7.1.0
- Cloud provider: AWS (us-west-2)
- Cluster configuration:

| Node type | Node size | Node quantity | Node storage |
|:----------|:----------|:----------|:----------|
| TiDB | 16 vCPU, 32 GiB | 2 | N/A |
| TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB |

### Parameter configuration

> **Note:**
>
> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help.

The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests based on different workloads with the following `prefill-for-recycle` configuration:

- For the `oltp_point_select` workload, use the default value of the [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) parameter:

```yaml
raft-engine.prefill-for-recycle = false
```

- For `oltp_insert`, `oltp_read_write` , `oltp_update_index`, and `oltp_update_non_index` workloads, enable the [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) parameter:

```yaml
raft-engine.prefill-for-recycle = true
```

### Benchmark executor

The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows:

- Machine type: Amazon EC2 (us-west-2)
- Instance type: c6a.2xlarge
- Sysbench version: sysbench 1.0.20 (using bundled LuaJIT 2.1.0-beta2)

## Test steps

This section introduces how to perform the Sysbench performance test step by step.

1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Dedicated cluster that meets the [test environment](#tidb-cluster) requirements.

For more information, see [Create a TiDB cluster](/tidb-cloud/create-tidb-cluster.md).

2. On the benchmark executor, connect to the newly created cluster and create a database named `sbtest`.

To connect to the cluster, see [Connect via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md).

To create the `sbtest` database, execute the following SQL statement:

```sql
CREATE DATABASE sbtest;
```

3. Load Sysbench data to the `sbtest` database.

1. The test in this document is implemented based on [sysbench](https://github.com/akopytov/sysbench). To install sysbench, see [Building and installing from source](https://github.com/akopytov/sysbench#building-and-installing-from-source).

2. Run the following `sysbench prepare` command to import 32 tables and 10,000,000 rows to the `sbtest` database. Replace `${HOST}`, `${PORT}`, `${THREAD}`, and `${PASSWORD}` with your actual values.

```shell
sysbench oltp_common \
--threads=${THREAD} \
--db-driver=mysql \
--mysql-db=sbtest \
--mysql-host=${HOST} \
--mysql-port=${PORT} \
--mysql-user=root \
--mysql-password=${PASSWORD} \
prepare --tables=32 --table-size=10000000
```

4. Run the following `sysbench run` command to conduct Sysbench performance tests on different workloads. This document conducts tests on five workloads: `oltp_point_select`, `oltp_read_write`, `oltp_update_non_index`, `oltp_update_index`, and `oltp_insert`. For each workload, this document conducts three tests with the `${THREAD}` value of `100`, `200`, and `400`. For each concurrency, the test takes 20 minutes.

```shell
sysbench ${WORKLOAD} run \
--mysql-host=${HOST} \
--mysql-port=${PORT} \
--mysql-user=root \
--db-driver=mysql \
--mysql-db=sbtest \
--threads=${THREAD} \
--time=1200 \
--report-interval=10 \
--tables=32 \
--table-size=10000000 \
--mysql-ignore-errors=1062,2013,8028,9007 \
--auto-inc=false \
--mysql-password=${PASSWORD}
```

## Test results

This section introduces the Sysbench performance of v7.1.0 in the [test environment](#test-environment).

### Point select performance

The performance on the `oltp_point_select` workload is as follows:

| Threads | TPS | 95% latency (ms)|
|:--------|:----------|:----------|
| 100 | 56,039 | 2.34 |
| 200 | 95,908 | 2.78 |
| 400 | 111,810 | 5.57 |

![Sysbench point select performance](/media/tidb-cloud/v7.1.0-oltp_select_point.png)

### Read write performance

The performance on the `oltp_read_write` workload is as follows:

| Threads | TPS | 95% latency (ms)|
|:--------|:----------|:----------|
| 100 | 1,789 | 66.8 |
| 200 | 2,842 | 97.6 |
| 400 | 3,090 | 191 |

![Sysbench read write performance](/media/tidb-cloud/v.7.1.0-oltp_read_write.png)

### Update non-index performance

The performance on the `oltp_update_non_index` workload is as follows:

| Threads | TPS | 95% latency (ms)|
|:--------|:----------|:----------|
| 100 | 7,944 | 16.7 |
| 200 | 13,844 | 19.0 |
| 400 | 29,063 | 20.4 |

![Sysbench update non-index performance](/media/tidb-cloud/v7.1.0-oltp_update_non_index.png)

### Update index performance

The performance on the `oltp_update_index` workload is as follows:

| Threads | TPS | 95% latency (ms)|
|:--------|:----------|:----------|
| 100 | 6,389 | 20 |
| 200 | 12,583 | 22.3 |
| 400 | 22,393 | 25.7 |

![Sysbench update index performance](/media/tidb-cloud/v7.1.0-oltp_update_index.png)

### Insert performance

The performance on the `oltp_insert` workload is as follows:

| Threads | TPS | 95% latency (ms)|
|:--------|:----------|:----------|
| 100 | 7,671 | 17.3 |
| 200 | 13,584 | 19.7 |
| 400 | 31,252 | 20 |

![Sysbench insert performance](/media/tidb-cloud/v7.1.0-oltp_insert.png)