Decouple the node cooperation ability and batch management ability of the edgeapplication

[WillardHu]

What would you like to be added/modified:

1. EdgeApplication can be overrides deployment spec via the node group or node label selector.
2. Fix the issue of closed-loop flow control.

Why is this needed:

EdgeApplication can be overrides deployment spec(i.e. replicas, image, commands and environments) via the node group, and pod traffics are closed-loop in a node group(Deployments managed by EdgeApplication share a Service). But in the real scenario, the scope of nodes that need batch operations is different from that of nodes that need to collaborate with each other.

For example, in the scenario of a smart park, there are many smart parks in every cities, we need application traffics are closed-loop in ones smart park, but the scope of app batch management may be city level or province level.

Therefore, we need to have a solution to decouple these.

Relevant issue #5669

Expected Outcomes:

1. Create a proposol of this iessue's solution.
2. Finish the coding. EdgeApplication can be overrides deployment spec via the node label selector.
3. The closed-loop flow control feature is bug-free.

Recommended Skills:

1. Understand K8s and KubeEdge.
2. Go language foundation.

Useful links:

• Node Group Management Proposal

[peonix0]

Looking forward to work on this project!

[SargamPuram]

Hi @WillardHu ,

I'm excited about the opportunity to contribute to the "Decouple Node Cooperation and Batch Management in EdgeApplication" project. My name is Sargam, and I have experience in Kubernetes and Go. Here’s my proposed approach:

1. I will create a comprehensive proposal addressing the solution for decoupling node cooperation ability and batch management ability in EdgeApplication. This proposal will involve a thorough analysis of the current architecture and the specific challenges posed by coupling node cooperation with batch management. I will outline the modifications required to introduce flexibility in deployment specifications, considering both node group and node label selectors. This document will serve as a blueprint for the subsequent implementation phase.

2. Implement Node Group/Label Selector Overrides:

I plan to enhance the EdgeApplication controller to support deployment spec overrides based on node group or node label selectors. This involves:

1. Modifying the controller logic to dynamically adjust replicas, images, commands, and environment variables according to the node group/label selectors.

2. Ensuring that the EdgeApplication can handle diverse deployment configurations efficiently, thus providing tailored deployment strategies based on specific node characteristics.

3. Implementing a mechanism to validate and apply these overrides seamlessly within the existing EdgeApplication framework.

4. Fix Closed-Loop Flow Control:

I will focus on resolving the closed-loop flow control issue to ensure that pod traffic remains within the designated node groups. This includes:

1. Analyzing the current flow control mechanisms to identify bottlenecks and points of failure.
2. Implementing robust traffic routing rules that enforce closed-loop behavior, ensuring that deployments managed by EdgeApplication share services correctly within a node group.
3. Conducting extensive testing to validate that the traffic control improvements are effective and do not introduce new issues.
   By decoupling these functionalities, we can achieve more precise control over deployment strategies, catering to both collaborative and batch management requirements effectively.

Looking forward to your feedback and the way forward to contributing! Also just curious are there any pretests for this?

Best Regards,
Sargam

[WillardHu]

Hi Sargam, I appreciate your interest in my project, your proposed plan is looks good. For implementing closed-loop flow control, you may explore the Kubernetes EndpointSlice feature for its capabilities and integration. Ultimately, I will conduct a thorough evaluation to make an informed decision on the candidates. Best regards, Willard Hu

…

________________________________ 发件人: Sargam Puram ***@***.***> 发送时间: 2024年8月4日 20:35 收件人: kubeedge/kubeedge ***@***.***> 抄送: wei hu ***@***.***>; Mention ***@***.***> 主题: Re: [kubeedge/kubeedge] Decouple the node cooperation ability and batch management ability of the edgeapplication (Issue #5755) Hi @WillardHu<https://github.com/WillardHu> , I'm excited about the opportunity to contribute to the "Decouple Node Cooperation and Batch Management in EdgeApplication" project. My name is Sargam, and I have experience in Kubernetes and Go. Here’s my proposed approach: 1. I will create a comprehensive proposal addressing the solution for decoupling node cooperation ability and batch management ability in EdgeApplication. This proposal will involve a thorough analysis of the current architecture and the specific challenges posed by coupling node cooperation with batch management. I will outline the modifications required to introduce flexibility in deployment specifications, considering both node group and node label selectors. This document will serve as a blueprint for the subsequent implementation phase. 2. Implement Node Group/Label Selector Overrides: I plan to enhance the EdgeApplication controller to support deployment spec overrides based on node group or node label selectors. This involves: 1. Modifying the controller logic to dynamically adjust replicas, images, commands, and environment variables according to the node group/label selectors. 2. Ensuring that the EdgeApplication can handle diverse deployment configurations efficiently, thus providing tailored deployment strategies based on specific node characteristics. 3. Implementing a mechanism to validate and apply these overrides seamlessly within the existing EdgeApplication framework. 4. Fix Closed-Loop Flow Control: I will focus on resolving the closed-loop flow control issue to ensure that pod traffic remains within the designated node groups. This includes: 1. Analyzing the current flow control mechanisms to identify bottlenecks and points of failure. 2. Implementing robust traffic routing rules that enforce closed-loop behavior, ensuring that deployments managed by EdgeApplication share services correctly within a node group. 3. Conducting extensive testing to validate that the traffic control improvements are effective and do not introduce new issues. By decoupling these functionalities, we can achieve more precise control over deployment strategies, catering to both collaborative and batch management requirements effectively. Looking forward to your feedback and the way forward to contributing! Also just curious are there any pretests for this? Best, Sargam ― Reply to this email directly, view it on GitHub<#5755 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AA6YUOOQAKVTHMOWYQYMXNDZPYNYNAVCNFSM6AAAAABLQDQGJCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRXGUZDQMBRGE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[SargamPuram]

Hi @WillardHu,

Thank you for the prompt feedback and for considering my proposal. I appreciate your insights on the EndpointSlice feature for implementing closed-loop flow control. I will explore this feature and evaluate its capabilities and integration potential for our use case.

I am excited to begin working on the comprehensive proposal and implementation plan. Please let me know if there are any specific requirements or additional details you would like me to include in the proposal. Additionally, if there are any pretests or initial tasks that I should complete, please share those as well.

Looking forward to contributing to this project.

Best regards,
Sargam

[magichan]

hi,I am very interested in this topic. I am familiar with Golang and have development experience with K8S Operators. At the same time, my current work is based on KubeEdge, so I want to gain a deeper understanding of KubeEdge through this topic. Therefore, I have the ability and motivation to complete this topic.

Having conducted research on K8S federation, I can understand the need for overriding deployment based on node label or node group.

Regarding the issue of closed-loop flow control. I've taken a preliminary look, and I don't quite understand the problem. Is the structure of the Node Group Management Proposal causing the existing issues? If so, I'll go through it in detail to see if I can understand the problem.

[EraKin575]

Hi, @WillardHu ,
I am Tejas, a final year UG student pursuing Computer Science. I am very excited to contribute to this project. I have thoroughly reviewed the project proposal and all of its supporting docs.Additionally , I have hands-on experience in Go and Kubernetes which I gained from my previous Software Development Internship. I have also prepared a high level solution plan and would really appreciate if you could review it: https://github.com/EraKin575/kubeedge/blob/proposal-plan/docs/proposals/edgeapplication-proposal.md

[EraKin575]

Thank you so much for considering me for this project @WillardHu !!!

[WillardHu]

Thank you so much for considering me for this project @WillardHu !!!

Let's successfully complete it. 😊

[EraKin575]

Thank you so much for considering me for this project @WillardHu !!!

Let's successfully complete it. 😊

Sure thing! Please provide me directions for moving ahead.Let's get started

[magichan]

@EraKin575 Could you assign yourself to this issue to facilitate my tracking of which issues are being handled by someone and which ones are unassigned, so that I can address them? Thank you very much.

[EraKin575]

/assign