Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BP-3.1][minor][docs] Rectify names of CDC sources for Flink and improve the content directory #3316

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ full database synchronization, sharding table synchronization, schema evolution
5. View job execution status through Flink WebUI or downstream database.

Try it out yourself with our more detailed [tutorial](docs/content/docs/get-started/quickstart/mysql-to-doris.md).
You can also see [connector overview](docs/content/docs/connectors/overview.md) to view a comprehensive catalog of the
You can also see [connector overview](docs/content/docs/connectors/pipeline-connectors/overview.md) to view a comprehensive catalog of the
connectors currently provided and understand more detailed configurations.


Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Legacy Flink CDC Sources
title: Flink Source 连接器
bookCollapseSection: true
weight: 5
weight: 2
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "DataStream API 打包指南"
weight: 998
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/datastream-api-package-guidance
- /connectors/flink-sources/datastream-api-package-guidance
---

<!--
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Db2"
weight: 9
weight: 7
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/db2-cdc
- /connectors/flink-sources/db2-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "MongoDB"
weight: 2
weight: 6
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/mongodb-cdc
- /connectors/flink-sources/mongodb-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "MySQL"
weight: 7
weight: 2
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/mysql-cdc
- /connectors/flink-sources/mysql-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "OceanBase"
weight: 4
weight: 9
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/oceanbase-cdc
- /connectors/flink-sources/oceanbase-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down Expand Up @@ -234,7 +234,7 @@ Flink SQL> CREATE TABLE orders (
);
```

您也可以访问 Flink CDC 官网文档,快速体验将数据从 OceanBase 导入到 Elasticsearch。更多信息,参考 [Flink CDC 官网文档](https://nightlies.apache.org/flink/flink-cdc-docs-release-3.0/docs/connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial/)。
您也可以访问 Flink CDC 官网文档,快速体验将数据从 OceanBase 导入到 Elasticsearch。更多信息,参考 [Flink CDC 官网文档]({{< ref "docs/connectors/flink-sources/tutorials/oceanbase-tutorial" >}})。

OceanBase CDC 连接器选项
----------------
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Oracle"
weight: 5
weight: 3
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/oracle-cdc
- /connectors/flink-sources/oracle-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Postgres"
weight: 6
weight: 5
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/postgres-cdc
- /connectors/flink-sources/postgres-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "SQL Server"
weight: 7
weight: 4
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/sqlserver-cdc
- /connectors/flink-sources/sqlserver-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "TiDB"
weight: 8
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tidb-cdc
- /connectors/flink-sources/tidb-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Legacy CDC Sources 教程"
title: "Flink CDC Sources 教程"
bookCollapseSection: true
weight: 999
---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "使用 Flink CDC 构建实时数据湖"
weight: 998
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/build-real-time-data-lake-tutorial.html
- /connectors/flink-sources/tutorials/build-real-time-data-lake-tutorial.html

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "使用 Flink CDC 构建 Streaming ETL"
weight: 999
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mysql-postgres-tutorial.html
- /connectors/flink-sources/tutorials/mysql-postgres-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Db2 教程"
weight: 2
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/db2-tutorial.html
- /connectors/flink-sources/tutorials/db2-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "MongoDB 教程"
weight: 1
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.html
- /connectors/flink-sources/tutorials/mongodb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "OceanBase 教程"
weight: 3
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial.html
- /connectors/flink-sources/tutorials/oceanbase-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Oracle 教程"
weight: 4
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/oracle-tutorial.html
- /connectors/flink-sources/tutorials/oracle-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "PolarDB-X 教程"
weight: 5
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.html
- /connectors/flink-sources/tutorials/mongodb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "SqlServer 教程"
weight: 6
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/sqlserver-tutorial.html
- /connectors/flink-sources/tutorials/sqlserver-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "TiDB 教程"
weight: 7
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/tutorials/tidb-tutorial.html
- /connectors/flink-sources/tutorials/tidb-tutorial.html
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Vitess"
weight: 10
type: docs
aliases:
- /connectors/legacy-flink-cdc-sources/vitess-cdc
- /connectors/flink-sources/vitess-cdc
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
46 changes: 0 additions & 46 deletions docs/content.zh/docs/connectors/overview.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Legacy CDC Sources
title: Pipeline 连接器
bookCollapseSection: true
weight: 5
weight: 1
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Doris"
weight: 3
weight: 5
type: docs
aliases:
- /connectors/doris
- /connectors/pipeline-connectors/doris
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Kafka"
weight: 5
weight: 4
type: docs
aliases:
- /connectors/kafka
- /connectors/pipeline-connectors/kafka
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "MySQL"
weight: 2
type: docs
aliases:
- /connectors/mysql
- /connectors/pipeline-connectors/mysql
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
45 changes: 45 additions & 0 deletions docs/content.zh/docs/connectors/pipeline-connectors/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: "概览"
weight: 1
type: docs
aliases:
- /connectors/pipeline-connectors/overview
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Pipeline Connectors

Flink CDC 提供了可用于 YAML 作业的 Pipeline Source 和 Sink 连接器来与外部系统交互。您可以直接使用这些连接器,只需将 JAR 文件添加到您的 Flink CDC 环境中,并在您的 YAML Pipeline 定义中指定所需的连接器。

## Supported Connectors

| 连接器 | 类型 | 支持的外部系统 |
|--------------------------------------------------------------------------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Apache Doris]({{< ref "docs/connectors/pipeline-connectors/doris" >}}) | Sink | <li> [Apache Doris](https://doris.apache.org/): 1.2.x, 2.x.x |
| [Kafka]({{< ref "docs/connectors/pipeline-connectors/kafka" >}}) | Sink | <li> [Kafka](https://kafka.apache.org/) |
| [MySQL]({{< ref "docs/connectors/pipeline-connectors/mysql" >}}) | Source | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 |
| [Paimon]({{< ref "docs/connectors/pipeline-connectors/paimon" >}}) | Sink | <li> [Paimon](https://paimon.apache.org/): 0.6, 0.7, 0.8 |
| [StarRocks]({{< ref "docs/connectors/pipeline-connectors/starrocks" >}}) | Sink | <li> [StarRocks](https://www.starrocks.io/): 2.x, 3.x |

## Develop Your Own Connector

如果现有的连接器无法满足您的需求,您可以自行开发自己的连接器,以将您的外部系统集成到 Flink CDC 数据管道中。查阅 [Flink CDC APIs]({{< ref "docs/developer-guide/understand-flink-cdc-api" >}}) 了解如何开发您自己的连接器。

{{< top >}}
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Paimon"
weight: 6
weight: 3
type: docs
aliases:
- /connectors/paimon
- /connectors/pipeline-connectors/paimon
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "StarRocks"
weight: 4
weight: 6
type: docs
aliases:
- /connectors/starrocks
- /connectors/pipeline-connectors/starrocks
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
Expand Down