From 917f81f9e98c7ae502734ad180dfbdab20bd7ad8 Mon Sep 17 00:00:00 2001 From: Jianyuan Jiang Date: Thu, 7 Dec 2023 15:11:30 +0800 Subject: [PATCH 1/8] Add temp.md --- temp.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 temp.md diff --git a/temp.md b/temp.md new file mode 100644 index 0000000000000..af27ff4986a7b --- /dev/null +++ b/temp.md @@ -0,0 +1 @@ +This is a test file. \ No newline at end of file From c8f07d7d12fbffa19e66bd7be93075c843cc4be7 Mon Sep 17 00:00:00 2001 From: Jianyuan Jiang Date: Thu, 7 Dec 2023 15:11:36 +0800 Subject: [PATCH 2/8] Delete temp.md --- temp.md | 1 - 1 file changed, 1 deletion(-) delete mode 100644 temp.md diff --git a/temp.md b/temp.md deleted file mode 100644 index af27ff4986a7b..0000000000000 --- a/temp.md +++ /dev/null @@ -1 +0,0 @@ -This is a test file. \ No newline at end of file From e83af658c1d137aae74a0643781dd121d1f879cf Mon Sep 17 00:00:00 2001 From: jiangjianyuan Date: Thu, 7 Dec 2023 15:16:49 +0800 Subject: [PATCH 3/8] add cdc redo configuration --- ticdc/ticdc-changefeed-config.md | 3 +++ ticdc/ticdc-open-api-v2.md | 20 +++++++++++--------- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/ticdc/ticdc-changefeed-config.md b/ticdc/ticdc-changefeed-config.md index f1cf0d869aa89..3fc36ee435902 100644 --- a/ticdc/ticdc-changefeed-config.md +++ b/ticdc/ticdc-changefeed-config.md @@ -220,6 +220,9 @@ flush-worker-num = 8 # The behavior to compress redo log files. # Available options are "" and "lz4". The default value is "", which means no compression. compression = "" +# The concurrency for uploading a single redo file. +# The default value is 1, which means concurrency is disabled. +flush-concurrency = 1 [integrity] # Whether to enable the checksum validation for single-row data. The default value is "none", which means to disable the feature. Value options are "none" and "correctness". diff --git a/ticdc/ticdc-open-api-v2.md b/ticdc/ticdc-open-api-v2.md index d3509cc36e341..207070cabaa42 100644 --- a/ticdc/ticdc-open-api-v2.md +++ b/ticdc/ticdc-open-api-v2.md @@ -281,15 +281,17 @@ The descriptions of the `replica_config` parameters are as follows. The `consistent` parameters are described as follows: -| Parameter name | Description | -|:-----------------|:---------------------------------------| -| `flush_interval` | `UINT64` type. The interval to flush redo log files. (Optional) | -| `level` | `STRING` type. The consistency level of the replicated data. (Optional) | -| `max_log_size` | `UINT64` type. The maximum value of redo log. (Optional) | -| `storage` | `STRING` type. The destination address of the storage. (Optional) | -| `use_file_backend` | `BOOL` type. Specifies whether to store the redo log in a local file. (Optional) | -| `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) | -| `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) | +| Parameter name | Description | +|:----------------------|:--------------------------------------------------------------------------------------------------------| +| `flush_interval` | `UINT64` type. The interval to flush redo log files. (Optional) | +| `level` | `STRING` type. The consistency level of the replicated data. (Optional) | +| `max_log_size` | `UINT64` type. The maximum value of redo log. (Optional) | +| `storage` | `STRING` type. The destination address of the storage. (Optional) | +| `use_file_backend` | `BOOL` type. Specifies whether to store the redo log in a local file. (Optional) | +| `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) | +| `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) | +| `compression` | `STRING` type. The behavior to compress redo log files, available options are "" and "lz4".. (Optional) | +| `flush_concurrency` | `INT` type. The concurrency for uploading a single file. (Optional) | The `filter` parameters are described as follows: From d242516a2ee46047353b026407b847f6c4afe1ba Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 7 Dec 2023 17:39:14 +0800 Subject: [PATCH 4/8] rollback the table format changes --- ticdc/ticdc-open-api-v2.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/ticdc/ticdc-open-api-v2.md b/ticdc/ticdc-open-api-v2.md index 207070cabaa42..785513cb750bf 100644 --- a/ticdc/ticdc-open-api-v2.md +++ b/ticdc/ticdc-open-api-v2.md @@ -281,15 +281,15 @@ The descriptions of the `replica_config` parameters are as follows. The `consistent` parameters are described as follows: -| Parameter name | Description | -|:----------------------|:--------------------------------------------------------------------------------------------------------| -| `flush_interval` | `UINT64` type. The interval to flush redo log files. (Optional) | -| `level` | `STRING` type. The consistency level of the replicated data. (Optional) | -| `max_log_size` | `UINT64` type. The maximum value of redo log. (Optional) | -| `storage` | `STRING` type. The destination address of the storage. (Optional) | -| `use_file_backend` | `BOOL` type. Specifies whether to store the redo log in a local file. (Optional) | -| `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) | -| `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) | +| Parameter name | Description | +|:-----------------|:---------------------------------------| +| `flush_interval` | `UINT64` type. The interval to flush redo log files. (Optional) | +| `level` | `STRING` type. The consistency level of the replicated data. (Optional) | +| `max_log_size` | `UINT64` type. The maximum value of redo log. (Optional) | +| `storage` | `STRING` type. The destination address of the storage. (Optional) | +| `use_file_backend` | `BOOL` type. Specifies whether to store the redo log in a local file. (Optional) | +| `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) | +| `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) | | `compression` | `STRING` type. The behavior to compress redo log files, available options are "" and "lz4".. (Optional) | | `flush_concurrency` | `INT` type. The concurrency for uploading a single file. (Optional) | From 760201104125f6869276b34d8c758d79e07557af Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 7 Dec 2023 17:40:05 +0800 Subject: [PATCH 5/8] Update ticdc/ticdc-open-api-v2.md --- ticdc/ticdc-open-api-v2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-open-api-v2.md b/ticdc/ticdc-open-api-v2.md index 785513cb750bf..aea1e909b056c 100644 --- a/ticdc/ticdc-open-api-v2.md +++ b/ticdc/ticdc-open-api-v2.md @@ -290,7 +290,7 @@ The `consistent` parameters are described as follows: | `use_file_backend` | `BOOL` type. Specifies whether to store the redo log in a local file. (Optional) | | `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) | | `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) | -| `compression` | `STRING` type. The behavior to compress redo log files, available options are "" and "lz4".. (Optional) | +| `compression` | `STRING` type. The behavior to compress redo log files. Available options are `""` and `"lz4"`. The default value is `""`, which means no compression. (Optional) | | `flush_concurrency` | `INT` type. The concurrency for uploading a single file. (Optional) | The `filter` parameters are described as follows: From 0c9e881334e1457b57cb42173fc1cdb197959c26 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 7 Dec 2023 17:50:21 +0800 Subject: [PATCH 6/8] Update ticdc/ticdc-open-api-v2.md --- ticdc/ticdc-open-api-v2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-open-api-v2.md b/ticdc/ticdc-open-api-v2.md index aea1e909b056c..9592c154210e7 100644 --- a/ticdc/ticdc-open-api-v2.md +++ b/ticdc/ticdc-open-api-v2.md @@ -291,7 +291,7 @@ The `consistent` parameters are described as follows: | `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) | | `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) | | `compression` | `STRING` type. The behavior to compress redo log files. Available options are `""` and `"lz4"`. The default value is `""`, which means no compression. (Optional) | -| `flush_concurrency` | `INT` type. The concurrency for uploading a single file. (Optional) | +| `flush_concurrency` | `INT` type. The concurrency for uploading a single file. The default value is `1`, which means concurrency is disabled(Optional) | The `filter` parameters are described as follows: From 45df2305715e9a4f459bea1f35e2206dc5f4e63d Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 7 Dec 2023 17:50:36 +0800 Subject: [PATCH 7/8] Update ticdc/ticdc-open-api-v2.md --- ticdc/ticdc-open-api-v2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-open-api-v2.md b/ticdc/ticdc-open-api-v2.md index 9592c154210e7..ddc9a82b22294 100644 --- a/ticdc/ticdc-open-api-v2.md +++ b/ticdc/ticdc-open-api-v2.md @@ -291,7 +291,7 @@ The `consistent` parameters are described as follows: | `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) | | `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) | | `compression` | `STRING` type. The behavior to compress redo log files. Available options are `""` and `"lz4"`. The default value is `""`, which means no compression. (Optional) | -| `flush_concurrency` | `INT` type. The concurrency for uploading a single file. The default value is `1`, which means concurrency is disabled(Optional) | +| `flush_concurrency` | `INT` type. The concurrency for uploading a single file. The default value is `1`, which means concurrency is disabled (Optional) | The `filter` parameters are described as follows: From b478dc66f8f2717e4fa9f045637d2d4b88b741a4 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 8 Dec 2023 11:10:41 +0800 Subject: [PATCH 8/8] Update ticdc/ticdc-open-api-v2.md --- ticdc/ticdc-open-api-v2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ticdc/ticdc-open-api-v2.md b/ticdc/ticdc-open-api-v2.md index ddc9a82b22294..d38f8cffef0a4 100644 --- a/ticdc/ticdc-open-api-v2.md +++ b/ticdc/ticdc-open-api-v2.md @@ -291,7 +291,7 @@ The `consistent` parameters are described as follows: | `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) | | `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) | | `compression` | `STRING` type. The behavior to compress redo log files. Available options are `""` and `"lz4"`. The default value is `""`, which means no compression. (Optional) | -| `flush_concurrency` | `INT` type. The concurrency for uploading a single file. The default value is `1`, which means concurrency is disabled (Optional) | +| `flush_concurrency` | `INT` type. The concurrency for uploading a single file. The default value is `1`, which means concurrency is disabled. (Optional) | The `filter` parameters are described as follows: