Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
848e1a6
update-doc
qqqdan Oct 23, 2025
00c129a
Update setup-aws-self-hosted-kafka-private-link-service.md
qiancai Oct 24, 2025
36181cf
delete setup-aws-self-hosted-kafka-private-link-service-premium.md as…
qiancai Oct 27, 2025
7dcd7c4
Apply suggestions from code review
qiancai Oct 27, 2025
32a6655
wrap ali content with custom content and make minor wording updates
qiancai Oct 27, 2025
7bd4f0e
Apply suggestions from code review
qiancai Oct 27, 2025
547a121
Update setup-aws-self-hosted-kafka-private-link-service.md
qiancai Oct 27, 2025
1bb459d
Merge branch 'privatelink-doc--premium' of https://github.com/pingcap…
qiancai Oct 27, 2025
092d87f
Update changefeed-overview.md
qiancai Oct 27, 2025
5b88014
Update tidb-cloud/changefeed-overview.md
qiancai Oct 28, 2025
195d9dd
Delete tidb-cloud/changefeed-overview-premium.md
qiancai Oct 28, 2025
d7b12c1
Update changefeed-overview.md
qiancai Oct 27, 2025
93ec858
Delete tidb-cloud/changefeed-sink-to-mysql-premium.md
qiancai Oct 28, 2025
222dbf2
Merge branch 'privatelink-doc--premium' of https://github.com/pingcap…
qiancai Oct 28, 2025
6a948c3
Update changefeed-sink-to-apache-kafka.md
qiancai Oct 28, 2025
94def80
Delete changefeed-sink-to-apache-kafka-premium.md
qiancai Oct 28, 2025
9a724ae
split Changefeed Billing into two docs
qiancai Oct 28, 2025
2b2def9
add the cost sections
qiancai Oct 28, 2025
7a10cec
Update tidb-cloud/changefeed-overview.md
qiancai Oct 28, 2025
ffa7d69
Apply suggestions from code review
qiancai Oct 28, 2025
02179a5
Update tidb-cloud-billing-ticdc-rcu.md
qiancai Oct 28, 2025
69444bb
Update tidb-cloud/changefeed-sink-to-apache-kafka.md
qiancai Oct 28, 2025
88c3d6d
Update changefeed-sink-to-apache-kafka.md
qiancai Oct 28, 2025
4f7bd56
Update changefeed-overview.md
qiancai Oct 28, 2025
9a3848f
replace with TiDB Cloud Premium with {{{ .premium }}}
qiancai Oct 28, 2025
61b2ebb
Merge branch 'release-8.5' into privatelink-doc--premium
qiancai Oct 28, 2025
333c54b
move the two premium docs to the premium folder
qiancai Oct 28, 2025
2627833
Update TOC-tidb-cloud-premium.md
qiancai Oct 28, 2025
431e7a7
fix two doc links
qiancai Oct 28, 2025
9598c3c
fix broken links and a role name
qiancai Oct 29, 2025
74f0145
Merge branch 'release-8.5' into privatelink-doc--premium
qiancai Oct 29, 2025
d15565d
fix a wrong link
qiancai Oct 29, 2025
fa1973f
Update setup-aws-self-hosted-kafka-private-link-service.md
qiancai Oct 29, 2025
39b4242
fix a broken link
qiancai Oct 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions TOC-tidb-cloud-premium.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,13 @@
- [CSV Configurations for Importing Data](/tidb-cloud/csv-config-for-import-data.md)
- [Troubleshoot Access Denied Errors during Data Import from Amazon S3](/tidb-cloud/troubleshoot-import-access-denied-error.md)
- [Connect AWS DMS to TiDB Cloud clusters](/tidb-cloud/tidb-cloud-connect-aws-dms.md)
- Stream Data
- [Changefeed Overview](/tidb-cloud/changefeed-overview.md)
- [To MySQL Sink](/tidb-cloud/changefeed-sink-to-mysql.md)
- [To Kafka Sink](/tidb-cloud/changefeed-sink-to-apache-kafka.md)
- Reference
- [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md)
- [Set Up Private Endpoint for Changefeeds](/tidb-cloud/premium/set-up-sink-private-endpoint-premium.md)
- Security
- [Security Overview](/tidb-cloud/security-overview.md)
- Identity Access Control
Expand All @@ -243,6 +250,7 @@
- [Credits](/tidb-cloud/tidb-cloud-billing.md#credits)
- [Payment Method Setting](/tidb-cloud/tidb-cloud-billing.md#payment-method)
- [Billing from Cloud Provider Marketplace](/tidb-cloud/tidb-cloud-billing.md#billing-from-cloud-provider-marketplace)
- [Billing for Changefeed](/tidb-cloud/premium/tidb-cloud-billing-ticdc-ccu.md)
- [Manage Budgets](/tidb-cloud/tidb-cloud-budget.md)
- Integrations
- [Airbyte](/tidb-cloud/integrate-tidbcloud-with-airbyte.md)
Expand Down
40 changes: 32 additions & 8 deletions tidb-cloud/changefeed-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,21 @@ TiDB Cloud changefeed helps you stream data from TiDB Cloud to other data servic

> **Note:**
>
> - Currently, TiDB Cloud only allows up to 100 changefeeds per cluster.
> - Currently, TiDB Cloud only allows up to 100 changefeeds per <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
> - Currently, TiDB Cloud only allows up to 100 table filter rules per changefeed.
> - For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.

## View the Changefeed page

To access the changefeed feature, take the following steps:

1. In the [TiDB Cloud console](https://tidbcloud.com), navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project.
1. In the [TiDB Cloud console](https://tidbcloud.com), <CustomContent plan="dedicated">navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project.</CustomContent><CustomContent plan="premium">navigate to the [**TiDB Instances**](https://tidbcloud.com/tidbs) page.</CustomContent>

> **Tip:**
>
> You can use the combo box in the upper-left corner to switch between organizations, projects, and clusters.

2. Click the name of your target cluster to go to its overview page, and then click **Data** > **Changefeed** in the left navigation pane. The changefeed page is displayed.
2. Click the name of your target <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent> to go to its overview page, and then click **Data** > **Changefeed** in the left navigation pane. The changefeed page is displayed.

On the **Changefeed** page, you can create a changefeed, view a list of existing changefeeds, and operate the existing changefeeds (such as scaling, pausing, resuming, editing, and deleting a changefeed).

Expand All @@ -36,22 +36,46 @@ To create a changefeed, refer to the tutorials:
- [Sink to TiDB Cloud](/tidb-cloud/changefeed-sink-to-tidb-cloud.md)
- [Sink to cloud storage](/tidb-cloud/changefeed-sink-to-cloud-storage.md)

## Query Changefeed RCUs
## Query changefeed capacity

<CustomContent plan="dedicated">

For TiDB Cloud Dedicated, you can query the TiCDC Replication Capacity Units (RCUs) of a changefeed.

1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
2. Locate the corresponding changefeed you want to query, and click **...** > **View** in the **Action** column.
3. You can see the current TiCDC Replication Capacity Units (RCUs) in the **Specification** area of the page.

</CustomContent>
<CustomContent plan="premium">

For {{{ .premium }}}, you can query the TiCDC Changefeed Capacity Units (CCUs) of a changefeed.

1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB instance.
2. Locate the corresponding changefeed you want to query, and click **...** > **View** in the **Action** column.
3. You can see the current TiCDC Changefeed Capacity Units (CCUs) in the **Specification** area of the page.

</CustomContent>

## Scale a changefeed

<CustomContent plan="dedicated">

You can change the TiCDC Replication Capacity Units (RCUs) of a changefeed by scaling up or down the changfeed.

> **Note:**
>
> - To scale a changefeed for a cluster, make sure that all changefeeds for this cluster are created after March 28, 2023.
> - If a cluster has changefeeds created before March 28, 2023, neither the existing changefeeds nor newly created changefeeds for this cluster support scaling up or down.

1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
</CustomContent>
<CustomContent plan="premium">

You can change the TiCDC Changefeed Capacity Units (CCUs) of a changefeed by scaling up or down the changfeed.

</CustomContent>

1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
2. Locate the corresponding changefeed you want to scale, and click **...** > **Scale Up/Down** in the **Action** column.
3. Select a new specification.
4. Click **Submit**.
Expand All @@ -60,7 +84,7 @@ It takes about 10 minutes to complete the scaling process (during which the chan

## Pause or resume a changefeed

1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
2. Locate the corresponding changefeed you want to pause or resume, and click **...** > **Pause/Resume** in the **Action** column.

## Edit a changefeed
Expand All @@ -69,7 +93,7 @@ It takes about 10 minutes to complete the scaling process (during which the chan
>
> TiDB Cloud currently only allows editing changefeeds in the paused status.

1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
2. Locate the changefeed you want to pause, and click **...** > **Pause** in the **Action** column.
3. When the changefeed status changes to `Paused`, click **...** > **Edit** to edit the corresponding changefeed.

Expand All @@ -84,7 +108,7 @@ It takes about 10 minutes to complete the scaling process (during which the chan

## Delete a changefeed

1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>.
2. Locate the corresponding changefeed you want to delete, and click **...** > **Delete** in the **Action** column.

## Changefeed billing
Expand Down
63 changes: 58 additions & 5 deletions tidb-cloud/changefeed-sink-to-apache-kafka.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,31 @@ summary: This document explains how to create a changefeed to stream data from T

This document describes how to create a changefeed to stream data from TiDB Cloud to Apache Kafka.

<CustomContent plan="dedicated">

> **Note:**
>
> - To use the changefeed feature, make sure that your TiDB Cloud Dedicated cluster version is v6.1.3 or later.
> - For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.

</CustomContent>
<CustomContent plan="premium">

> **Note:**
>
> For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.

</CustomContent>

## Restrictions

- For each TiDB Cloud cluster, you can create up to 100 changefeeds.
- For each TiDB Cloud <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>, you can create up to 100 changefeeds.
- Currently, TiDB Cloud does not support uploading self-signed TLS certificates to connect to Kafka brokers.
- Because TiDB Cloud uses TiCDC to establish changefeeds, it has the same [restrictions as TiCDC](https://docs.pingcap.com/tidb/stable/ticdc-overview#unsupported-scenarios).
- If the table to be replicated does not have a primary key or a non-null unique index, the absence of a unique constraint during replication could result in duplicated data being inserted downstream in some retry scenarios.

<CustomContent plan="dedicated">

- If you choose Private Link or Private Service Connect as the network connectivity method, ensure that your TiDB cluster version meets the following requirements:

- For v6.5.x: version v6.5.9 or later
Expand All @@ -30,6 +44,8 @@ This document describes how to create a changefeed to stream data from TiDB Clou
- If you want to distribute changelogs by primary key or index value to Kafka partition with a specified index name, make sure the version of your TiDB cluster is v7.5.0 or later.
- If you want to distribute changelogs by column value to Kafka partition, make sure the version of your TiDB cluster is v7.5.0 or later.

</CustomContent>

## Prerequisites

Before creating a changefeed to stream data to Apache Kafka, you need to complete the following prerequisites:
Expand All @@ -39,12 +55,14 @@ Before creating a changefeed to stream data to Apache Kafka, you need to complet

### Network

Ensure that your TiDB cluster can connect to the Apache Kafka service. You can choose one of the following connection methods:
Ensure that your TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent> can connect to the Apache Kafka service. You can choose one of the following connection methods:

- Private Connect: ideal for avoiding VPC CIDR conflicts and meeting security compliance, but incurs additional [Private Data Link Cost](/tidb-cloud/tidb-cloud-billing-ticdc-rcu.md#private-data-link-cost).
- VPC Peering: suitable as a cost-effective option, but requires managing potential VPC CIDR conflicts and security considerations.
- Public IP: suitable for a quick setup.

<CustomContent plan="dedicated">

<SimpleTab>
<div label="Private Connect">

Expand Down Expand Up @@ -87,6 +105,35 @@ It is **NOT** recommended to use Public IP in a production environment.

</div>
</SimpleTab>
</CustomContent>

<CustomContent plan="premium">

<SimpleTab>
<div label="Private Connect">

Private Connect leverages **Private Link** or **Private Service Connect** technologies from cloud providers to enable resources in your VPC to connect to services in other VPCs using private IP addresses, as if those services were hosted directly within your VPC.

TiDB Cloud currently supports Private Connect only for self-hosted Kafka. It does not support direct integration with MSK, Confluent Kafka, or other Kafka SaaS services. To connect to these Kafka SaaS services via Private Connect, you can deploy a [kafka-proxy](https://github.com/grepplabs/kafka-proxy) as an intermediary, effectively exposing the Kafka service as self-hosted Kafka.

If your Apache Kafka service is hosted on AWS, follow [Set Up Self-Hosted Kafka Private Link Service in AWS](/tidb-cloud/setup-aws-self-hosted-kafka-private-link-service.md) to configure the network connection and obtain the **Bootstrap Ports** information, and then follow [Set Up Private Endpoint for Changefeeds](/tidb-cloud/premium/set-up-sink-private-endpoint-premium.md) to create a private endpoint.

</div>
<div label="Public IP">

If you want to provide Public IP access to your Apache Kafka service, assign Public IP addresses to all your Kafka brokers.

It is **NOT** recommended to use Public IP in a production environment.

</div>

<div label="VPC Peering">

Currently, the VPC Peering feature for {{{ .premium }}} instances is only available upon request. To request this feature, click **?** in the lower-right corner of the [TiDB Cloud console](https://tidbcloud.com) and click **Request Support**. Then, fill in "Apply for VPC Peering for {{{ .premium }}} instance" in the **Description** field and click **Submit**.

</div>
</SimpleTab>
</CustomContent>

### Kafka ACL authorization

Expand All @@ -100,7 +147,7 @@ For example, if your Kafka cluster is in Confluent Cloud, you can see [Resources
## Step 1. Open the Changefeed page for Apache Kafka

1. Log in to the [TiDB Cloud console](https://tidbcloud.com).
2. Navigate to the cluster overview page of the target TiDB cluster, and then click **Data** > **Changefeed** in the left navigation pane.
2. Navigate to the overview page of the target TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent>, and then click **Data** > **Changefeed** in the left navigation pane.
3. Click **Create Changefeed**, and select **Kafka** as **Destination**.

## Step 2. Configure the changefeed target
Expand Down Expand Up @@ -140,6 +187,8 @@ The steps vary depending on the connectivity method you select.
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.

</div>

<CustomContent plan="dedicated">
<div label="Private Service Connect (Google Cloud)">

1. In **Connectivity Method**, select **Private Service Connect**.
Expand All @@ -158,6 +207,9 @@ The steps vary depending on the connectivity method you select.
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.

</div>
</CustomContent>

<CustomContent plan="dedicated">
<div label="Private Link (Azure)">

1. In **Connectivity Method**, select **Private Link**.
Expand All @@ -176,6 +228,7 @@ The steps vary depending on the connectivity method you select.
11. Return to the [TiDB Cloud console](https://tidbcloud.com) to confirm that you have accepted the connection request. TiDB Cloud will test the connection and proceed to the next page if the test succeeds.

</div>
</CustomContent>
</SimpleTab>

## Step 3. Set the changefeed
Expand Down Expand Up @@ -219,7 +272,7 @@ The steps vary depending on the connectivity method you select.
6. If you select **Avro** as your data format, you will see some Avro-specific configurations on the page. You can fill in these configurations as follows:

- In the **Decimal** and **Unsigned BigInt** configurations, specify how TiDB Cloud handles the decimal and unsigned bigint data types in Kafka messages.
- In the **Schema Registry** area, fill in your schema registry endpoint. If you enable **HTTP Authentication**, the fields for user name and password are displayed and automatically filled in with your TiDB cluster endpoint and password.
- In the **Schema Registry** area, fill in your schema registry endpoint. If you enable **HTTP Authentication**, the fields for user name and password are displayed and automatically filled in with your TiDB <CustomContent plan="dedicated">cluster</CustomContent><CustomContent plan="premium">instance</CustomContent> endpoint and password.

7. In the **Topic Distribution** area, select a distribution mode, and then fill in the topic name configurations according to the mode.

Expand Down Expand Up @@ -272,7 +325,7 @@ The steps vary depending on the connectivity method you select.

## Step 4. Configure your changefeed specification

1. In the **Changefeed Specification** area, specify the number of Replication Capacity Units (RCUs) to be used by the changefeed.
1. In the **Changefeed Specification** area, specify the number of <CustomContent plan="dedicated">Replication Capacity Units (RCUs)</CustomContent><CustomContent plan="premium">Changefeed Capacity Units (CCUs)</CustomContent> to be used by the changefeed.
2. In the **Changefeed Name** area, specify a name for the changefeed.
3. Click **Next** to check the configurations you set and go to the next page.

Expand Down
Loading
Loading