[Umbrella][Feature] Flink Kubernetes Module V2

[Al-assad]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I had searched in the issues and found no similar issues.

Describe the proposal

Due to many issues with the current version of StreamPark Flink on Kubernetes in terms of submitting and tracking tasks, we hope to refactor the entire module based on the Flink Kubernetes Operator to better address the following problems:

• Inaccurate and confusing tracking of Flink on Kubernetes status;
• Unstable and lengthy Flink K8s Application Job submission process;
• Scattered Event Watcher thread pool code;

Resources

• Design documentation: https://www.craft.me/s/LrTTzYHqSOLYqY
• Demo code: https://github.com/Al-assad/streampark-flink-kubernetes-v2

Task list

• [Feature] [Flink-K8s-V2] Create shaded module for flink-kubernetes-operator-api dependency #2880 @Al-assad
• [Feature] [Flink-K8s-V2] Port the new module codes to Streampark-flink-kubernetes-v2 #2881 @Al-assad
• [Feature] [Flink-K8s-V2] Refactor the lifecycle control of Flink application-mode jobs on Kubernetes #2882 @caicancai @Al-assad
• [Feature] [Flink-K8s-V2] Refactor the lifecycle control of Flink session-mode jobs on Kubernetes #2883 @caicancai
• [Feature] [Flink-K8s-V2] Refactor the lifecycle control code of Flink clusters on Kubernetes #2884 @caicancai
• [Feature] [Flink-K8s-V2] Refactor the state tracking of Flink on Kubernetes #2885 @ChengJie1053 @Al-assad
• [Feature] [Flink-K8s-V2] Modify docker and helm: Adding SVC configuration related to embedded file server. #2886 @Al-assad
• [Feature] [Flink-K8s-V2] Remove the old streampark-flink-kubernetes module code #2887

Dependency relationship of subtasks

#2880 → #2881 → #2882, #2883, #2884, #2885 → #2887

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[Al-assad]

@ChengJie1053 @caicancai
Hey Bros, If you're totally stoked about tasks #2883 to #2887, they're all yours for the taking! Just claim 'em right here! 😆

[caicancai]

@Al-assad i pick #2883 and #2884