Skip to content

Add cdc redo configuration (#15619) #15627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Dec 8, 2023
Merged

Add cdc redo configuration (#15619) #15627

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8f20049
Add temp.md
sdojjy Dec 7, 2023
46ce516
Delete temp.md
sdojjy Dec 7, 2023
f8724fd
add cdc redo configuration
sdojjy Dec 7, 2023
7c467e8
rollback the table format changes
qiancai Dec 7, 2023
8b2ae0d
Update ticdc/ticdc-open-api-v2.md
qiancai Dec 7, 2023
9e3f958
Update ticdc/ticdc-open-api-v2.md
qiancai Dec 7, 2023
bd2c5c7
Update ticdc/ticdc-open-api-v2.md
qiancai Dec 7, 2023
caa84c3
Update ticdc/ticdc-open-api-v2.md
qiancai Dec 8, 2023
2bcc9b7
Update ticdc-sink-to-kafka.md
qiancai Dec 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ticdc/ticdc-changefeed-config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,9 @@ flush-worker-num = 8
# The behavior to compress redo log files.
# Available options are "" and "lz4". The default value is "", which means no compression.
compression = ""
# The concurrency for uploading a single redo file.
# The default value is 1, which means concurrency is disabled.
flush-concurrency = 1

# The following configuration items only take effect when the downstream is Kafka. Supported starting from v6.5.3.
[sink.kafka-config]
Expand Down
2 changes: 2 additions & 0 deletions ticdc/ticdc-open-api-v2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,8 @@ The `consistent` parameters are described as follows:
| `use_file_backend` | `BOOL` type. Specifies whether to store the redo log in a local file. (Optional) |
| `encoding_worker_num` | `INT` type. The number of encoding and decoding workers in the redo module. (Optional) |
| `flush_worker_num` | `INT` type. The number of flushing workers in the redo module. (Optional) |
| `compression` | `STRING` type. The behavior to compress redo log files. Available options are `""` and `"lz4"`. The default value is `""`, which means no compression. (Optional) |
| `flush_concurrency` | `INT` type. The concurrency for uploading a single file. The default value is `1`, which means concurrency is disabled. (Optional) |

The `filter` parameters are described as follows:

Expand Down
2 changes: 1 addition & 1 deletion ticdc/ticdc-sink-to-kafka.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ The following are examples when using Kafka SASL authentication:

The minimum set of permissions required for TiCDC to function properly is as follows.

- The `Create` and `Write` permissions for the Topic [resource type](https://docs.confluent.io/platform/current/kafka/authorization.html#resources).
- The `Create`, `Write`, and `Describe` permissions for the Topic [resource type](https://docs.confluent.io/platform/current/kafka/authorization.html#resources).
Copy link
Collaborator

@qiancai qiancai Dec 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Align with https://github.com/pingcap/docs-cn/pull/15623/files#r1419882281

- The `DescribeConfigs` permission for the Cluster resource type.

### Integrate TiCDC with Kafka Connect (Confluent Platform)
Expand Down