From 8df378d9060a3b699d4fc027858da3f0b984c02f Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:39:15 -0600 Subject: [PATCH 01/12] Reorganized Introduction (experiment) --- README.md | 5 ++-- architecture.md | 48 ++++++++++++++++++++++++++++++++++ features.md | 27 +++++++++++++++++++ overview.md | 69 ++----------------------------------------------- 4 files changed, 80 insertions(+), 69 deletions(-) create mode 100644 architecture.md create mode 100644 features.md diff --git a/README.md b/README.md index 32e25700094aa..36c8cf1b49a96 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,9 @@ ## Documentation List + About TiDB - - [TiDB Introduction](overview.md#tidb-introduction) - - [TiDB Architecture](overview.md#tidb-architecture) + - [TiDB Introduction](overview.md) + - [TiDB Architecture](architecture.md) + - [TiDB Key Features](features.md) + Quick Start - [TiDB Quick Start Guide](QUICKSTART.md) - [Basic SQL Statements](try-tidb.md) diff --git a/architecture.md b/architecture.md new file mode 100644 index 0000000000000..c20e116d84bf0 --- /dev/null +++ b/architecture.md @@ -0,0 +1,48 @@ +--- +title: TiDB Architecture +summary: Learn about what TiDB is, and the key features, architecture and roadmap of TiDB. +category: introduction +--- + +# TiDB architecture + +To better understand TiDB's features, you need to understand the TiDB architecture. The TiDB cluster includes three key components: the TiDB server, the PD server, and the TiKV server. In addition, TiDB also provides the [TiSpark](https://github.com/pingcap/tispark/) component for the complex OLAP requirements. + +![image alt text](media/tidb-architecture.png) + +### TiDB server + +The TiDB server is in charge of the following operations: + +1. Receiving the SQL requests + +2. Processing the SQL related logics + +3. Locating the TiKV address for storing and computing data through Placement Driver (PD) + +4. Exchanging data with TiKV + +5. Returning the result + +The TiDB server is stateless. It does not store data and it is for computing only. TiDB is horizontally scalable and provides the unified interface to the outside through the load balancing components such as Linux Virtual Server (LVS), HAProxy, or F5. + +### Placement Driver server + +The Placement Driver (PD) server is the managing component of the entire cluster and is in charge of the following three operations: + +1. Storing the metadata of the cluster such as the region location of a specific key. + +2. Scheduling and load balancing regions in the TiKV cluster, including but not limited to data migration and Raft group leader transfer. + +3. Allocating the transaction ID that is globally unique and monotonic increasing. + +As a cluster, PD needs to be deployed to an odd number of nodes. Usually it is recommended to deploy to 3 online nodes at least. + +### TiKV server + +The TiKV server is responsible for storing data. From an external view, TiKV is a distributed transactional Key-Value storage engine. Region is the basic unit to store data. Each Region stores the data for a particular Key Range which is a left-closed and right-open interval from StartKey to EndKey. There are multiple Regions in each TiKV node. TiKV uses the Raft protocol for replication to ensure the data consistency and disaster recovery. The replicas of the same Region on different nodes compose a Raft Group. The load balancing of the data among different TiKV nodes are scheduled by PD. Region is also the basic unit for scheduling the load balance. + +### TiSpark + +TiSpark deals with the complex OLAP requirements. TiSpark makes Spark SQL directly run on the storage layer of the TiDB cluster, combines the advantages of the distributed TiKV cluster, and integrates into the big data ecosystem. With TiSpark, TiDB can support both OLTP and OLAP scenarios in one cluster, so the users never need to worry about data synchronization. + diff --git a/features.md b/features.md new file mode 100644 index 0000000000000..2210031c1a94d --- /dev/null +++ b/features.md @@ -0,0 +1,27 @@ +--- +title: TiDB Features +summary: Learn about what TiDB is, and the key features, architecture and roadmap of TiDB. +category: introduction +--- + +# Features + +### Horizontal scalability + +Horizontal scalability is the most important feature of TiDB. The scalability includes two aspects: the computing capability and the storage capacity. The TiDB server processes the SQL requests. As the business grows, the overall processing capability and higher throughput can be achieved by simply adding more TiDB server nodes. Data is stored in TiKV. As the size of the data grows, the scalability of data can be resolved by adding more TiKV server nodes. PD schedules data in Regions among the TiKV nodes and migrates part of the data to the newly added node. So in the early stage, you can deploy only a few service instances. For example, it is recommended to deploy at least 3 TiKV nodes, 3 PD nodes and 2 TiDB nodes. As business grows, more TiDB and TiKV instances can be added on-demand. + +### High availability + +High availability is another important feature of TiDB. All of the three components, TiDB, TiKV and PD, can tolerate the failure of some instances without impacting the availability of the entire cluster. For each component, See the following for more details about the availability, the consequence of a single instance failure and how to recover. + +#### TiDB + +TiDB is stateless and it is recommended to deploy at least two instances. The front-end provides services to the outside through the load balancing components. If one of the instances is down, the Session on the instance will be impacted. From the application’s point of view, it is a single request failure but the service can be regained by reconnecting to the TiDB server. If a single instance is down, the service can be recovered by restarting the instance or by deploying a new one. + +#### PD + +PD is a cluster and the data consistency is ensured using the Raft protocol. If an instance is down but the instance is not a Raft Leader, there is no impact on the service at all. If the instance is a Raft Leader, a new Leader will be elected to recover the service. During the election which is approximately 3 seconds, PD cannot provide service. It is recommended to deploy three instances. If one of the instances is down, the service can be recovered by restarting the instance or by deploying a new one. + +#### TiKV + +TiKV is a cluster and the data consistency is ensured using the Raft protocol. The number of the replicas can be configurable and the default is 3 replicas. The load of TiKV servers are balanced through PD. If one of the node is down, all the Regions in the node will be impacted. If the failed node is the Leader of the Region, the service will be interrupted and a new election will be initiated. If the failed node is a Follower of the Region, the service will not be impacted. If a TiKV node is down for a period of time (default 30 minutes), PD will move the data to another TiKV node. diff --git a/overview.md b/overview.md index 6cc4d2063e5d4..709cc29b3ca99 100644 --- a/overview.md +++ b/overview.md @@ -1,12 +1,10 @@ --- -title: About TiDB +title: TiDB Introduction summary: Learn about what TiDB is, and the key features, architecture and roadmap of TiDB. category: introduction --- -# About TiDB - -## TiDB introduction +# TiDB introduction TiDB (The pronunciation is: /'taɪdiːbi:/ tai-D-B, etymology: titanium) is an open-source distributed scalable Hybrid Transactional and Analytical Processing (HTAP) database. It features infinite horizontal scalability, strong consistency, and high availability. TiDB is MySQL compatible and serves as a one-stop data warehouse for both OLTP (Online Transactional Processing) and OLAP (Online Analytical Processing) workloads. @@ -53,66 +51,3 @@ Read the [Roadmap](https://github.com/pingcap/docs/blob/master/ROADMAP.md). - **Stack Overflow**: https://stackoverflow.com/questions/tagged/tidb - **Mailing list**: [Google Group](https://groups.google.com/forum/#!forum/tidb-user) -## TiDB architecture - -To better understand TiDB's features, you need to understand the TiDB architecture. The TiDB cluster includes three key components: the TiDB server, the PD server, and the TiKV server. In addition, TiDB also provides the [TiSpark](https://github.com/pingcap/tispark/) component for the complex OLAP requirements. - -![image alt text](media/tidb-architecture.png) - -### TiDB server - -The TiDB server is in charge of the following operations: - -1. Receiving the SQL requests - -2. Processing the SQL related logics - -3. Locating the TiKV address for storing and computing data through Placement Driver (PD) - -4. Exchanging data with TiKV - -5. Returning the result - -The TiDB server is stateless. It does not store data and it is for computing only. TiDB is horizontally scalable and provides the unified interface to the outside through the load balancing components such as Linux Virtual Server (LVS), HAProxy, or F5. - -### Placement Driver server - -The Placement Driver (PD) server is the managing component of the entire cluster and is in charge of the following three operations: - -1. Storing the metadata of the cluster such as the region location of a specific key. - -2. Scheduling and load balancing regions in the TiKV cluster, including but not limited to data migration and Raft group leader transfer. - -3. Allocating the transaction ID that is globally unique and monotonic increasing. - -As a cluster, PD needs to be deployed to an odd number of nodes. Usually it is recommended to deploy to 3 online nodes at least. - -### TiKV server - -The TiKV server is responsible for storing data. From an external view, TiKV is a distributed transactional Key-Value storage engine. Region is the basic unit to store data. Each Region stores the data for a particular Key Range which is a left-closed and right-open interval from StartKey to EndKey. There are multiple Regions in each TiKV node. TiKV uses the Raft protocol for replication to ensure the data consistency and disaster recovery. The replicas of the same Region on different nodes compose a Raft Group. The load balancing of the data among different TiKV nodes are scheduled by PD. Region is also the basic unit for scheduling the load balance. - -### TiSpark - -TiSpark deals with the complex OLAP requirements. TiSpark makes Spark SQL directly run on the storage layer of the TiDB cluster, combines the advantages of the distributed TiKV cluster, and integrates into the big data ecosystem. With TiSpark, TiDB can support both OLTP and OLAP scenarios in one cluster, so the users never need to worry about data synchronization. - -## Features - -### Horizontal scalability - -Horizontal scalability is the most important feature of TiDB. The scalability includes two aspects: the computing capability and the storage capacity. The TiDB server processes the SQL requests. As the business grows, the overall processing capability and higher throughput can be achieved by simply adding more TiDB server nodes. Data is stored in TiKV. As the size of the data grows, the scalability of data can be resolved by adding more TiKV server nodes. PD schedules data in Regions among the TiKV nodes and migrates part of the data to the newly added node. So in the early stage, you can deploy only a few service instances. For example, it is recommended to deploy at least 3 TiKV nodes, 3 PD nodes and 2 TiDB nodes. As business grows, more TiDB and TiKV instances can be added on-demand. - -### High availability - -High availability is another important feature of TiDB. All of the three components, TiDB, TiKV and PD, can tolerate the failure of some instances without impacting the availability of the entire cluster. For each component, See the following for more details about the availability, the consequence of a single instance failure and how to recover. - -#### TiDB - -TiDB is stateless and it is recommended to deploy at least two instances. The front-end provides services to the outside through the load balancing components. If one of the instances is down, the Session on the instance will be impacted. From the application’s point of view, it is a single request failure but the service can be regained by reconnecting to the TiDB server. If a single instance is down, the service can be recovered by restarting the instance or by deploying a new one. - -#### PD - -PD is a cluster and the data consistency is ensured using the Raft protocol. If an instance is down but the instance is not a Raft Leader, there is no impact on the service at all. If the instance is a Raft Leader, a new Leader will be elected to recover the service. During the election which is approximately 3 seconds, PD cannot provide service. It is recommended to deploy three instances. If one of the instances is down, the service can be recovered by restarting the instance or by deploying a new one. - -#### TiKV - -TiKV is a cluster and the data consistency is ensured using the Raft protocol. The number of the replicas can be configurable and the default is 3 replicas. The load of TiKV servers are balanced through PD. If one of the node is down, all the Regions in the node will be impacted. If the failed node is the Leader of the Region, the service will be interrupted and a new election will be initiated. If the failed node is a Follower of the Region, the service will not be impacted. If a TiKV node is down for a period of time (default 30 minutes), PD will move the data to another TiKV node. From 7930c1941d130722c56b8d4c8fb821b10f03d2fd Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:41:17 -0600 Subject: [PATCH 02/12] Update features.md --- features.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/features.md b/features.md index 2210031c1a94d..52f4361d73c0c 100644 --- a/features.md +++ b/features.md @@ -1,27 +1,27 @@ --- title: TiDB Features -summary: Learn about what TiDB is, and the key features, architecture and roadmap of TiDB. +summary: A list of some of the key features of the TiDB Database category: introduction --- -# Features +# TiDB Features -### Horizontal scalability +## Horizontal scalability Horizontal scalability is the most important feature of TiDB. The scalability includes two aspects: the computing capability and the storage capacity. The TiDB server processes the SQL requests. As the business grows, the overall processing capability and higher throughput can be achieved by simply adding more TiDB server nodes. Data is stored in TiKV. As the size of the data grows, the scalability of data can be resolved by adding more TiKV server nodes. PD schedules data in Regions among the TiKV nodes and migrates part of the data to the newly added node. So in the early stage, you can deploy only a few service instances. For example, it is recommended to deploy at least 3 TiKV nodes, 3 PD nodes and 2 TiDB nodes. As business grows, more TiDB and TiKV instances can be added on-demand. -### High availability +## High availability High availability is another important feature of TiDB. All of the three components, TiDB, TiKV and PD, can tolerate the failure of some instances without impacting the availability of the entire cluster. For each component, See the following for more details about the availability, the consequence of a single instance failure and how to recover. -#### TiDB +### TiDB TiDB is stateless and it is recommended to deploy at least two instances. The front-end provides services to the outside through the load balancing components. If one of the instances is down, the Session on the instance will be impacted. From the application’s point of view, it is a single request failure but the service can be regained by reconnecting to the TiDB server. If a single instance is down, the service can be recovered by restarting the instance or by deploying a new one. -#### PD +### PD PD is a cluster and the data consistency is ensured using the Raft protocol. If an instance is down but the instance is not a Raft Leader, there is no impact on the service at all. If the instance is a Raft Leader, a new Leader will be elected to recover the service. During the election which is approximately 3 seconds, PD cannot provide service. It is recommended to deploy three instances. If one of the instances is down, the service can be recovered by restarting the instance or by deploying a new one. -#### TiKV +### TiKV TiKV is a cluster and the data consistency is ensured using the Raft protocol. The number of the replicas can be configurable and the default is 3 replicas. The load of TiKV servers are balanced through PD. If one of the node is down, all the Regions in the node will be impacted. If the failed node is the Leader of the Region, the service will be interrupted and a new election will be initiated. If the failed node is a Follower of the Region, the service will not be impacted. If a TiKV node is down for a period of time (default 30 minutes), PD will move the data to another TiKV node. From 8dc0b44724936309c1a8646a2c7ea5ea7bfefc2c Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:42:00 -0600 Subject: [PATCH 03/12] Update overview.md --- overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/overview.md b/overview.md index 709cc29b3ca99..6d4bdb0d9496e 100644 --- a/overview.md +++ b/overview.md @@ -1,6 +1,6 @@ --- title: TiDB Introduction -summary: Learn about what TiDB is, and the key features, architecture and roadmap of TiDB. +summary: An introduction to TiDB and some of the key benefits it provides. category: introduction --- From def246d022ef133a4e9f067abc45c398a67c9785 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:42:11 -0600 Subject: [PATCH 04/12] Update overview.md --- overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/overview.md b/overview.md index 6d4bdb0d9496e..74fb3b0ce80a5 100644 --- a/overview.md +++ b/overview.md @@ -1,6 +1,6 @@ --- title: TiDB Introduction -summary: An introduction to TiDB and some of the key benefits it provides. +summary: An introduction to TiDB and the key benefits it provides. category: introduction --- From 46f0053153ca868ec3331454d8bf31c28eb2a598 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:42:27 -0600 Subject: [PATCH 05/12] Update overview.md --- overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/overview.md b/overview.md index 74fb3b0ce80a5..6ebaaf406566a 100644 --- a/overview.md +++ b/overview.md @@ -4,7 +4,7 @@ summary: An introduction to TiDB and the key benefits it provides. category: introduction --- -# TiDB introduction +# TiDB Introduction TiDB (The pronunciation is: /'taɪdiːbi:/ tai-D-B, etymology: titanium) is an open-source distributed scalable Hybrid Transactional and Analytical Processing (HTAP) database. It features infinite horizontal scalability, strong consistency, and high availability. TiDB is MySQL compatible and serves as a one-stop data warehouse for both OLTP (Online Transactional Processing) and OLAP (Online Analytical Processing) workloads. From aea7c524f553a7bb778026e42446e961bd48975d Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:43:03 -0600 Subject: [PATCH 06/12] Update architecture.md --- architecture.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/architecture.md b/architecture.md index c20e116d84bf0..91829d92214e9 100644 --- a/architecture.md +++ b/architecture.md @@ -1,10 +1,10 @@ --- title: TiDB Architecture -summary: Learn about what TiDB is, and the key features, architecture and roadmap of TiDB. +summary: The key architecture components of the TiDB Database Platform. category: introduction --- -# TiDB architecture +# TiDB Architecture To better understand TiDB's features, you need to understand the TiDB architecture. The TiDB cluster includes three key components: the TiDB server, the PD server, and the TiKV server. In addition, TiDB also provides the [TiSpark](https://github.com/pingcap/tispark/) component for the complex OLAP requirements. From d26ab4ab5cd982c8956f3a204aaf84a22f685660 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:43:27 -0600 Subject: [PATCH 07/12] Update architecture.md --- architecture.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/architecture.md b/architecture.md index 91829d92214e9..c6186a7087f73 100644 --- a/architecture.md +++ b/architecture.md @@ -10,7 +10,7 @@ To better understand TiDB's features, you need to understand the TiDB architectu ![image alt text](media/tidb-architecture.png) -### TiDB server +## TiDB server The TiDB server is in charge of the following operations: @@ -26,7 +26,7 @@ The TiDB server is in charge of the following operations: The TiDB server is stateless. It does not store data and it is for computing only. TiDB is horizontally scalable and provides the unified interface to the outside through the load balancing components such as Linux Virtual Server (LVS), HAProxy, or F5. -### Placement Driver server +## Placement Driver server The Placement Driver (PD) server is the managing component of the entire cluster and is in charge of the following three operations: @@ -38,11 +38,10 @@ The Placement Driver (PD) server is the managing component of the entire cluster As a cluster, PD needs to be deployed to an odd number of nodes. Usually it is recommended to deploy to 3 online nodes at least. -### TiKV server +## TiKV server The TiKV server is responsible for storing data. From an external view, TiKV is a distributed transactional Key-Value storage engine. Region is the basic unit to store data. Each Region stores the data for a particular Key Range which is a left-closed and right-open interval from StartKey to EndKey. There are multiple Regions in each TiKV node. TiKV uses the Raft protocol for replication to ensure the data consistency and disaster recovery. The replicas of the same Region on different nodes compose a Raft Group. The load balancing of the data among different TiKV nodes are scheduled by PD. Region is also the basic unit for scheduling the load balance. -### TiSpark +## TiSpark TiSpark deals with the complex OLAP requirements. TiSpark makes Spark SQL directly run on the storage layer of the TiDB cluster, combines the advantages of the distributed TiKV cluster, and integrates into the big data ecosystem. With TiSpark, TiDB can support both OLTP and OLAP scenarios in one cluster, so the users never need to worry about data synchronization. - From 87675b0bb3d2c1f2ec39b77d1e0b938a551ecff6 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:47:48 -0600 Subject: [PATCH 08/12] Update features.md --- features.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/features.md b/features.md index 52f4361d73c0c..291cb995605f4 100644 --- a/features.md +++ b/features.md @@ -1,10 +1,10 @@ --- -title: TiDB Features +title: TiDB Key Features summary: A list of some of the key features of the TiDB Database category: introduction --- -# TiDB Features +# TiDB Key Features ## Horizontal scalability From 1b89353f92366c0e31354f2dda2f1c63100a7baf Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:52:29 -0600 Subject: [PATCH 09/12] Update architecture.md --- architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/architecture.md b/architecture.md index c6186a7087f73..298612be68826 100644 --- a/architecture.md +++ b/architecture.md @@ -1,6 +1,6 @@ --- title: TiDB Architecture -summary: The key architecture components of the TiDB Database Platform. +summary: The key architecture components of the TiDB Database Platform category: introduction --- From 91ff0e98fe23f205a9bf6916221a8ba7769b1cd2 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:53:15 -0600 Subject: [PATCH 10/12] Update overview.md --- overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/overview.md b/overview.md index 6ebaaf406566a..948562c7cd5f1 100644 --- a/overview.md +++ b/overview.md @@ -1,6 +1,6 @@ --- title: TiDB Introduction -summary: An introduction to TiDB and the key benefits it provides. +summary: An introduction to the TiDB database platform category: introduction --- From e4d314931815710f3ae0140b501d607c9db194cc Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:53:39 -0600 Subject: [PATCH 11/12] Update architecture.md --- architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/architecture.md b/architecture.md index 298612be68826..68a8e4fe1b25d 100644 --- a/architecture.md +++ b/architecture.md @@ -1,6 +1,6 @@ --- title: TiDB Architecture -summary: The key architecture components of the TiDB Database Platform +summary: The key architecture components of the TiDB database platform category: introduction --- From bc7fb1d43737352b3690e7acfaeb5fd456f48df0 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 8 Sep 2018 14:54:27 -0600 Subject: [PATCH 12/12] Update features.md --- features.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/features.md b/features.md index 291cb995605f4..b1f4195dcbf4c 100644 --- a/features.md +++ b/features.md @@ -1,6 +1,6 @@ --- title: TiDB Key Features -summary: A list of some of the key features of the TiDB Database +summary: Key features of the TiDB database platform category: introduction ---