benchmark: add TPC-H benchmark result #443
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| --- | ||
| title: TiDB TPC-H 50G Performance Test Report V2.0 | ||
| category: benchmark | ||
| draft: true | ||
| --- | ||
|
|
||
| # TiDB TPC-H 50G Performance Test Report | ||
|
|
||
| ## Test purpose | ||
|
|
||
| The test aims at comparing the performance between TiDB v1.0 and v2.0 in OLAP scenario. | ||
|
There was a problem hiding this comment. -> This test aims to compare the performances of TiDB 1.0 and TiDB 2.0 in the OLAP scenario. |
||
|
|
||
| > **Note**: Different test environments may lead to different test results. | ||
|
|
||
|
|
||
| ## Test environment | ||
|
|
||
| ### Machine hardware information | ||
|
|
||
| | Type | Name | | ||
| |------------|------------------------------------------------------| | ||
| | OS | linux (CentOS 7.3.1611) | | ||
|
|
||
| | CPU | 40 vCPUs, Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz | | ||
| | RAM | 128GB | | ||
| | DISK | 1.5T SSD * 2 + Optane SSD * 1 | | ||
|
|
||
| ### TPC-H | ||
|
|
||
| [tidb-bench/tpch](https://github.com/pingcap/tidb-bench/tree/master/tpch) | ||
|
|
||
| ### Cluster topology | ||
|
|
||
| | Machine IP | Deployment Instance | | ||
| |--------------|---------------------| | ||
| | 172.16.31.2 | TiKV \* 2 | | ||
| | 172.16.31.3 | TiKV \* 2 | | ||
| | 172.16.31.6 | TiKV \* 2 | | ||
| | 172.16.31.8 | TiKV \* 2 | | ||
| | 172.16.31.10 | TiKV \* 2 | | ||
| | 172.16.31.10 | PD \* 1 | | ||
| | 172.16.31.4 | TiDB \* 1 | | ||
|
|
||
| ### Corresponding TiDB version information | ||
|
|
||
| TiDB 1.0： | ||
|
There was a problem hiding this comment. Use the colon for English text. |
||
|
|
||
| | Component | Version | Commit Hash | | ||
| |--------|-------------|--------------------------------------------| | ||
| | TiDB | v1.0.9 | 4c7ee3580cd0a69319b2c0c08abdc59900df7344 | | ||
| | TiKV | v1.0.8 | 2bb923a4cd23dbf68f0d16169fd526dc5c1a9f4a | | ||
| | PD | v1.0.8 | 137fa734472a76c509fbfd9cb9bc6d0dc804a3b7 | | ||
|
|
||
| TiDB 2.0： | ||
|
|
||
|
|
||
| | Component | Version | Commit Hash | | ||
| |--------|-------------|--------------------------------------------| | ||
| | TiDB | v2.0.0-rc.6 | 82d35f1b7f9047c478f4e1e82aa0002abc8107e7 | | ||
| | TiKV | v2.0.0-rc.6 | 8bd5c54966c6ef42578a27519bce4915c5b0c81f | | ||
| | PD | v2.0.0-rc.6 | 9b824d288126173a61ce7d51a71fc4cb12360201 | | ||
|
|
||
| ## Test result | ||
|
|
||
| | Query ID | TiDB 2.0 | TiDB 1.0 | | ||
| |-----------|--------------------|------------------| | ||
| | 1 | 33.915s | 215.305s | | ||
| | 2 | 25.575s | Nan | | ||
| | 3 | 59.631s | 196.003s | | ||
| | 4 | 30.234s | 249.919s | | ||
| | 5 | 31.666s | OOM | | ||
| | 6 | 13.111s | 118.709s | | ||
| | 7 | 31.710s | OOM | | ||
| | 8 | 31.734s | 800.546s | | ||
| | 9 | 34.211s | 630.639s | | ||
| | 10 | 30.774s | 133.547s | | ||
| | 11 | 27.692s | 78.026s | | ||
| | 12 | 27.962s | 124.641s | | ||
| | 13 | 27.676s | 174.695s | | ||
| | 14 | 19.676s | 110.602s | | ||
| | 15 | NaN | Nan | | ||
| | 16 | 24.890s | 40.529s | | ||
| | 17 | 245.796s | NaN | | ||
| | 18 | 91.256s | OOM | | ||
| | 19 | 37.615s | NaN | | ||
| | 20 | 44.167s | 212.201s | | ||
| | 21 | 31.466s | OOM | | ||
| | 22 | 31.539s | 125.471s | | ||
|
|
||
|  | ||
|
|
||
| It should be notified that: | ||
|
|
||
| - In the diagram above, the orange bars represent the query result of Release 1.0 and the blue bars represent the query rersult of Release 2.0. The ordinate shows the processing time of queries. | ||
|
There was a problem hiding this comment. The ordinate shows the processing time of queries, the shorter the better. ? There was a problem hiding this comment. maybe s/shows/represents/ is more suitable ? |
||
| - Both Queries 15 are tagged with Nan because both TiDB 1.0 and TiDB 2.0 do not support view. | ||
|
There was a problem hiding this comment. Query 15 is tagged with "NaN" because VIEW is not supported in both TiDB 1.0 and 2.0 releases. |
||
| - Queries 2, 17, and 19 in the TiDB 1.0 column are tagged with Nan because TiDB 1.0 does not return a result for long. | ||
| - Queries 5, 7, 18, and 21 in the TiDB 1.0 column are tagged with OOM because TiDB 1.0 memory space is occupied too much and then is killed by oom-killer. | ||
|
There was a problem hiding this comment. Queries 5, 7, 18, and 21 in the TiDB 1.0 column are tagged with "OOM" because the memory consumption is too high and tidb-server is killed by oom-killer. |
||
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Delete this line.