From 3e61204ed733cb40c91efc4ec010c06def0352f6 Mon Sep 17 00:00:00 2001 From: toutdesuite Date: Fri, 5 Jun 2020 10:20:59 +0800 Subject: [PATCH 1/4] Update ticdc-overview.md --- ticdc/ticdc-overview.md | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/ticdc/ticdc-overview.md b/ticdc/ticdc-overview.md index 2458704389789..8ed024dd477bf 100644 --- a/ticdc/ticdc-overview.md +++ b/ticdc/ticdc-overview.md @@ -7,6 +7,10 @@ aliases: ['/docs/dev/reference/tools/ticdc/overview/'] # TiCDC Overview +> **Note:** +> +> TiCDC is experimental. It is **not recommended** to use this feature in the production environment. + [TiCDC](https://github.com/pingcap/ticdc) is a tool for replicating the incremental data of TiDB. This tool is implemented by pulling TiKV change logs. It can restore data to a consistent state with any upstream TSO, and provides [TiCDC Open Protocol](/ticdc/ticdc-open-protocol.md) to support other systems to subscribe to data changes. ## TiCDC Architecture @@ -41,13 +45,29 @@ Currently, the TiCDC sink component supports replicating data to the following d - Databases compatible with MySQL protocol. The sink component provides the final consistency support. - Kafka based on the TiCDC Open Protocol. The sink component ensures the row-level order, final consistency or strict transactional consistency. -### Black and white table lists - -You can write blacklist and whitelist filtering rules to filter or only replicate all changed data in certain databases or tables. The filtering rules are similar to those of MySQL such as `replication-rules-db` or `replication-rules-table`. - ## Restrictions To replicate data to TiDB or MySQL, you must ensure that the following requirements are satisfied to guarantee data correctness: - The table to be replicated has the primary key or a unique index. - If the table to be replicated only has unique indexes, each column of at least one unique index is explicitly defined in the table schema as `NOT NULL`. + +### Unsupported scenarios + +Currently, the unsupported scenarios are as follows: + +- The replication of partition tables. +- TiKV clusters that use RawKV alone. +- The [new framework for collations](/character-set-and-collation.md#new-framework-for-collations) in TiDB v4.0. Before you enable this feature, make sure that the downstream cluster is the TiDB cluster with the same collation as the upstream; otherwise, the inconsistent collation leads to an issue that the data cannot be located. +- [The DDL operation `CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) and the [SEQUENCE function](/sql-statements/sql-statement-create-sequence.md#sequence-function) in TiDB v4.0. When the upstream TiDB uses SEQUENCE, TiCDC ignores SEQUENCE DDL operations/functions performed upstream. However, DML operations using SEQUENCE functions can be correctly replicated. +- The [TiKV Hibernate Region](https://github.com/tikv/tikv/blob/master/docs/reference/configuration/raftstore-config.md#hibernate-region). TiCDC prevents the Region from entering the hibernated state. +- The scheduling of existing replication tables to new TiCDC nodes, after the capacity of the TiCDC cluster is scaled out. +- The replication of Black and white table lists. + +## Manage TiCDC Cluster and Replication Tasks + +For details, see [Manage TiCDC Cluster and Replication Tasks](/ticdc/manage-ticdc.md). + +## TiCDC Open Protocol + +TiCDC Open Protocol is a row-level data change notification protocol that provides data sources for monitoring, caching, full-text indexing, analysis engines, and master-slave replication between different databases. TiCDC complies with TiCDC Open Protocol and replicates data changes of TiDB to third-party data medium such as MQ (Message Queue). For more information, see [TiCDC Open Protocol](/ticdc/ticdc-open-protocol.md). From f28a446c35d1eadf0d46ea2ae2674b5f407a915c Mon Sep 17 00:00:00 2001 From: toutdesuite Date: Fri, 5 Jun 2020 14:58:56 +0800 Subject: [PATCH 2/4] update wording --- ticdc/ticdc-overview.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ticdc/ticdc-overview.md b/ticdc/ticdc-overview.md index 8ed024dd477bf..2ac42592ab17a 100644 --- a/ticdc/ticdc-overview.md +++ b/ticdc/ticdc-overview.md @@ -54,11 +54,11 @@ To replicate data to TiDB or MySQL, you must ensure that the following requireme ### Unsupported scenarios -Currently, the unsupported scenarios are as follows: +Currently, The following scenarios are not supported: - The replication of partition tables. -- TiKV clusters that use RawKV alone. -- The [new framework for collations](/character-set-and-collation.md#new-framework-for-collations) in TiDB v4.0. Before you enable this feature, make sure that the downstream cluster is the TiDB cluster with the same collation as the upstream; otherwise, the inconsistent collation leads to an issue that the data cannot be located. +- The TiKV cluster that uses RawKV alone. +- The [new framework for collations](/character-set-and-collation.md#new-framework-for-collations) in TiDB v4.0. Before you enable this feature, make sure that the downstream cluster is the TiDB cluster with the same collation as those in the upstream; otherwise, inconsistent collations lead to the issue that the data cannot be located. - [The DDL operation `CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) and the [SEQUENCE function](/sql-statements/sql-statement-create-sequence.md#sequence-function) in TiDB v4.0. When the upstream TiDB uses SEQUENCE, TiCDC ignores SEQUENCE DDL operations/functions performed upstream. However, DML operations using SEQUENCE functions can be correctly replicated. - The [TiKV Hibernate Region](https://github.com/tikv/tikv/blob/master/docs/reference/configuration/raftstore-config.md#hibernate-region). TiCDC prevents the Region from entering the hibernated state. - The scheduling of existing replication tables to new TiCDC nodes, after the capacity of the TiCDC cluster is scaled out. From 1e50dc9ca4f25ab9f93c6580e08e65e4214a8352 Mon Sep 17 00:00:00 2001 From: Ran Date: Fri, 5 Jun 2020 21:20:54 +0800 Subject: [PATCH 3/4] Update ticdc/ticdc-overview.md --- ticdc/ticdc-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-overview.md b/ticdc/ticdc-overview.md index 2ac42592ab17a..df939f348cb58 100644 --- a/ticdc/ticdc-overview.md +++ b/ticdc/ticdc-overview.md @@ -59,7 +59,7 @@ Currently, The following scenarios are not supported: - The replication of partition tables. - The TiKV cluster that uses RawKV alone. - The [new framework for collations](/character-set-and-collation.md#new-framework-for-collations) in TiDB v4.0. Before you enable this feature, make sure that the downstream cluster is the TiDB cluster with the same collation as those in the upstream; otherwise, inconsistent collations lead to the issue that the data cannot be located. -- [The DDL operation `CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) and the [SEQUENCE function](/sql-statements/sql-statement-create-sequence.md#sequence-function) in TiDB v4.0. When the upstream TiDB uses SEQUENCE, TiCDC ignores SEQUENCE DDL operations/functions performed upstream. However, DML operations using SEQUENCE functions can be correctly replicated. +- The [DDL operation `CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) and the [SEQUENCE function](/sql-statements/sql-statement-create-sequence.md#sequence-function) in TiDB v4.0. When the upstream TiDB uses `SEQUENCE`, TiCDC ignores `SEQUENCE` DDL operations/functions performed upstream. However, DML operations using `SEQUENCE` functions can be correctly replicated. - The [TiKV Hibernate Region](https://github.com/tikv/tikv/blob/master/docs/reference/configuration/raftstore-config.md#hibernate-region). TiCDC prevents the Region from entering the hibernated state. - The scheduling of existing replication tables to new TiCDC nodes, after the capacity of the TiCDC cluster is scaled out. - The replication of Black and white table lists. From f4e5c7180071eb617458813af63f3e6688287fae Mon Sep 17 00:00:00 2001 From: toutdesuite Date: Fri, 5 Jun 2020 22:35:58 +0800 Subject: [PATCH 4/4] Update ticdc/ticdc-overview.md Co-authored-by: amyangfei --- ticdc/ticdc-overview.md | 1 - 1 file changed, 1 deletion(-) diff --git a/ticdc/ticdc-overview.md b/ticdc/ticdc-overview.md index df939f348cb58..a65bd1d0fe54d 100644 --- a/ticdc/ticdc-overview.md +++ b/ticdc/ticdc-overview.md @@ -56,7 +56,6 @@ To replicate data to TiDB or MySQL, you must ensure that the following requireme Currently, The following scenarios are not supported: -- The replication of partition tables. - The TiKV cluster that uses RawKV alone. - The [new framework for collations](/character-set-and-collation.md#new-framework-for-collations) in TiDB v4.0. Before you enable this feature, make sure that the downstream cluster is the TiDB cluster with the same collation as those in the upstream; otherwise, inconsistent collations lead to the issue that the data cannot be located. - The [DDL operation `CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) and the [SEQUENCE function](/sql-statements/sql-statement-create-sequence.md#sequence-function) in TiDB v4.0. When the upstream TiDB uses `SEQUENCE`, TiCDC ignores `SEQUENCE` DDL operations/functions performed upstream. However, DML operations using `SEQUENCE` functions can be correctly replicated.