Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

支持 FlinkDeployment 部署 #2930

Closed
1 of 2 tasks
potterhe opened this issue Dec 26, 2023 · 5 comments
Closed
1 of 2 tasks

支持 FlinkDeployment 部署 #2930

potterhe opened this issue Dec 26, 2023 · 5 comments
Labels
enhancement New feature or request

Comments

@potterhe
Copy link

potterhe commented Dec 26, 2023

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

Flink 社区提供了 flink-kubernetes-operator ,为在k8s中运行flink任务提供了全生命周期管理。为配合这个operator,社区的容器镜像flink:1.17.1 内置了非常多的脚本(路径在镜像内的/opt/flink/bin/)。当前通过flink-cdc.sh提交任务的方式,无法复用这种能力。

Solution

建议将flink-cdc的 pipiline 建设成一个普通的flink应用,通过flink本身提交任务的方式运行,这样就可以复用Flink社区的能力。

Alternatives

No response

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@potterhe potterhe added the enhancement New feature or request label Dec 26, 2023
@icchux
Copy link

icchux commented Dec 27, 2023

现在FlinkCDC-3.0整库同步的业务逻辑代码和提交Flink应用的代码应该分离开,而不是只支持remote和minicluster这两种模式。 我现在已经把FlinkCDC的 pipiline 改成一个普通的flink应用,然后写了一个提交Flink应用的客户端(支持Standalone、Yarn-session、YARN-application、k8s-application)等方式提交FlinkCDC的pipeline,用来整库同步,有需要的话可以创建一个pr

@fanqiejiang8 我也是需要这个,希望可以解耦分离,能通过一个普通的flink应用,提交到阿里云的实时计算平台,可以加个微信交流一下么

@fanqiejiang8
Copy link

@icchux 可以的啊 我加你

@icchux
Copy link

icchux commented Dec 27, 2023

@icchux 可以的啊 我加你

@fanqiejiang8 vx: ilycheeya

@edhe1988
Copy link

现在FlinkCDC-3.0整库同步的业务逻辑代码和提交Flink应用的代码应该分离开,而不是只支持remote和minicluster这两种模式。 我现在已经把FlinkCDC的 pipiline 改成一个普通的flink应用,然后写了一个提交Flink应用的客户端(支持Standalone、Yarn-session、YARN-application、k8s-application)等方式提交FlinkCDC的pipeline,用来整库同步,有需要的话可以创建一个pr

这个我也需要,要不你先上传到你的github,供我参考下。嘿嘿

@potterhe
Copy link
Author

#2934

@potterhe potterhe closed this as completed Jan 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants