Container Load Balancer - Design Doc #9014
Conversation
k8s-ci-robot
added
kind/feature
|
k8s-ci-robot
added
cncf-cla: no
|
Hi @georgeedward2000. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
the
size/M
✅ Deploy Preview for kubernetes-sigs-cloud-provide-azure ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: georgeedward2000 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
| linkTitle: "StandardV2 Load Balancer - Container Based Backendpool" | ||
| type: docs | ||
| description: > | ||
| Implement new Load Balancer sku - StandardV2 |
There was a problem hiding this comment.
This is great! Could you add a little more details on the doc, including benefits (compared to SLB), how it works (e.g. add a simple diagram to show its datapath), how to use (e.g. config and service spec sample)?
|
/release-note-none |
k8s-ci-robot
added
release-note-none
|
|
||
| - In its current form, the **Software Load Balancer (SLB)** only supports load balancing to nodes. In a container cluster environment, there are multiple processing units or pods on a single node. To ensure that a user's request reaches an application within a pod, the request is first sent to the SLB, which determines the most suitable node. At the node level, the traffic is further balanced among the multiple pods. This process may involve multiple hops between pods until the correct node is found, especially when externalTrafficPolicy=Cluster is set. Consequently, the traffic is load balanced multiple times before reaching the desired pod. | ||
|
|
||
| - In contrast, the **Container Load Balancer (CLB)** offers a fundamentally improved approach. The goal is to create an IP subnet overlay for all the pods that are part of a specific application. This means that when a user sends a request to the SLB, the SLB can directly route the traffic to the desired pod. This approach significantly reduces latency and provides a platform for new features, such as adding rule-based identities. In this context, the middleware capabilities of the cloud provider are utilized to construct and synchronize the Kubernetes cluster structure and its corresponding subnet overlay. |
There was a problem hiding this comment.
sends a reuqest to the SLB? typo?
There was a problem hiding this comment.
No, I am referring to the scenario where the user interacts with the application via the load balancer URL. For example, when the user sends an HTTP request (GET, POST, etc.) to the application using the load balancer URL, the request is distributed to one of the application's instances.
|
|
||
| 1. Current users do not need to take any action, and the ongoing changes will not affect them. | ||
|
|
||
| 2. New Users must create a Container Based Cluster. |
There was a problem hiding this comment.
Can you elaborate "Container Based Cluster"? How does users create such a cluster?
There was a problem hiding this comment.
This will require creating some resources on Azure side (similarly how now user needs RG, vnet, subnet, etc.). We will document that as well as provide reference implementation in CAPZ.
|
|
||
| 2. New Users must create a Container Based Cluster. | ||
|
|
||
| 3. Then, they must create a `Standard V2` sku Load Balancer |
There was a problem hiding this comment.
Will cloud provider azure automatically provision the LB when reconciling a svc? Or the user must create the LB manually?
There was a problem hiding this comment.
Right. The user only needs to initially configure the AKS instance to use CLB. No separate LB manual creation needed. I am updating the document.
| ### How to use | ||
|
|
||
| - Apart from selecting the correct SKU in Azure, no additional steps are required. | ||
| - Within the Kubernetes cluster, the pod, service, and all other resource manifests remain unchanged. T |
|
|
||
| - Above is the current datapath of SLB - Node based - Multiple networking hops. | ||
|
|
||
| - Bottom is the new proposal of SLB - Container based - Single networking hop. |
There was a problem hiding this comment.
For the bottom picture, does it mean there will be node IPs joining the backend pool of a CLB, instead of pod IPs?
There was a problem hiding this comment.
The backend pool will be comprised only of pod IPs, not node IPs
There was a problem hiding this comment.
Right, but for the picture, there are nodes instead of pods linking to the slb, so this makes me confused.
|
|
||
|  | ||
|
|
||
| ### Control Plane Batch Processing |
There was a problem hiding this comment.
Are svc create/update and endpoints update undergo the same code path of this batch processor? If so, ignore my comment. If not, better elaborate in the view of these workflows.
|
|
||
| 1. Introduces DiffTracker API to keep track of the synchronization between the Kubernetes (K8s) cluster and its NRP resources structure. | ||
|
|
||
| 2. Introduces LocationAndNRPServiceBatchUpdater as a parallel thread worker used as the main engine for DiffTracker synchronization. It runs continuously, waiting for updates in the cluster on a boolean channel and triggering NRP API requests. |
There was a problem hiding this comment.
Are there any differences between the existing endpoint slice updater?
There was a problem hiding this comment.
If you are referring to the endpoint slice informer which acts as an ES event consumer, the LocationAndNRPServiceBatchUpdater is just an extension to it. It does not replace it. If you are not referring to this one please elaborate.
|
|
||
| 3. All updates within the K8s cluster (regarding pods, services, egresses, etc.) are stored within DiffTracker. | ||
|
|
||
| 4. After each update, a boolean value of `true` is sent through the LocationAndNRPServiceBatchUpdater channel. |
There was a problem hiding this comment.
the ticker should also be added as a trigger. The update operation will be stalled if there is no enough requests
There was a problem hiding this comment.
Hmm, I am not sure I follow. Batch processor runs on separate thread, hence there won't be any stalling. Moreover, if the channel is already full, that is fine, since the batch processor will run at least one more time incorporating the latest changes (as difftracker works as a batch aggregator)
|
|
||
| - In its current form, the **Software Load Balancer (SLB)** only supports load balancing to nodes. In a container cluster environment, there are multiple processing units or pods on a single node. To ensure that a user's request reaches an application within a pod, the request is first sent to the SLB, which determines the most suitable node. At the node level, the traffic is further balanced among the multiple pods. This process may involve multiple hops between pods until the correct node is found, especially when externalTrafficPolicy=Cluster is set. Consequently, the traffic is load balanced multiple times before reaching the desired pod. | ||
|
|
||
| - In contrast, the **Container Load Balancer (CLB)** offers a fundamentally improved approach. The goal is to create an IP subnet overlay for all the pods that are part of a specific application. This means that when a user sends a request to the SLB, the SLB can directly route the traffic to the desired pod. This approach significantly reduces latency and provides a platform for new features, such as adding rule-based identities. In this context, the middleware capabilities of the cloud provider are utilized to construct and synchronize the Kubernetes cluster structure and its corresponding subnet overlay. |
There was a problem hiding this comment.
multiple subnet may co-exist in the same cluster. I'm wondering if there can be multiple container lb instances in the same cluster
There was a problem hiding this comment.
yes, it's expected there will be many, one per service of type LB
|
|
||
| 1. Current users do not need to take any action, and the ongoing changes will not affect them. | ||
|
|
||
| 2. New Users must create a Container Based Cluster. |
There was a problem hiding this comment.
This will require creating some resources on Azure side (similarly how now user needs RG, vnet, subnet, etc.). We will document that as well as provide reference implementation in CAPZ.
| 5. The LocationAndNRPServiceBatchUpdater run method waits to consume booleans from the channel. When a value is consumed, it uses all the updates stored in DiffTracker to update NRP using the NRP API. Eventually, all successful NRP API calls are stored back into DiffTracker to assert the equivalence of the two structures (K8s cluster and Azure). | ||
|
|
||
| 6. DiffTracker also functions as a batch operations aggregator. When a cluster undergoes multiple rapid updates, DiffTracker's state will continuously be updated. Meanwhile, the LocationAndNRPServiceBatchUpdater, running on a single thread, will consume updates from the channel. As a result, multiple updates can accumulate and be ready to be sent to the NRP in a single batch. | ||
|
|
There was a problem hiding this comment.
Maybe worth adding to the picture that worst case there can be another hop between the nodes before it reaches the pod (eg User->SLB->Node1->Node2->Pod4)
|
|
||
| - In its current form, the **Software Load Balancer (SLB)** only supports load balancing to nodes. In a container cluster environment, there are multiple processing units or pods on a single node. To ensure that a user's request reaches an application within a pod, the request is first sent to the SLB, which determines the most suitable node. At the node level, the traffic is further balanced among the multiple pods. This process may involve multiple hops between pods until the correct node is found, especially when externalTrafficPolicy=Cluster is set. Consequently, the traffic is load balanced multiple times before reaching the desired pod. | ||
|
|
||
| - In contrast, the **Container Load Balancer (CLB)** offers a fundamentally improved approach. The goal is to create an IP subnet overlay for all the pods that are part of a specific application. This means that when a user sends a request to the SLB, the SLB can directly route the traffic to the desired pod. This approach significantly reduces latency and provides a platform for new features, such as adding rule-based identities. In this context, the middleware capabilities of the cloud provider are utilized to construct and synchronize the Kubernetes cluster structure and its corresponding subnet overlay. |
There was a problem hiding this comment.
yes, it's expected there will be many, one per service of type LB
Co-authored-by: David Kowalski <50632861+david-kow@users.noreply.github.com>
Co-authored-by: David Kowalski <50632861+david-kow@users.noreply.github.com>
Co-authored-by: David Kowalski <50632861+david-kow@users.noreply.github.com>
Co-authored-by: David Kowalski <50632861+david-kow@users.noreply.github.com>
Co-authored-by: David Kowalski <50632861+david-kow@users.noreply.github.com>
What type of PR is this?
/kind feature
/kind design