Document role of cloudprovider healthchecks in L4 lb

[aaskey]

A load balancer type of service/deployment is created on GKE using the command below:

kubectl run my-nginx --image=nginx --port=80
kubectl expose deployment my-nginx --port=80 --type=LoadBalancer

A load balancer was created automatically. This is nice. However, from Google Cloud Platform/Networking/Load balancing, it complains:

This load balancer has no health check, so traffic will be sent to all instances regardless of their status.

Should the load balancer to be created with health check by default? I don't see an option to enable load balancer health check in service manifest. Would such an option be necessary in case we don't want to enable health check all the time?

[bprashanth]

kubelet does container level health checks so traffic doesn't go to unhealthy pods. If no pods are healthy, you (should) get an connection reset (RST).

If you use the source ip preservation feature (kubernetes/enhancements#27, alpha in 1.4) the service controller will program health checks for you so a node without endpoints doesn't get traffic.

We should probably surface this through docs.

[aaskey]

Retaining source ip info is very important for our applications. We are happy to see this source ip preservation feature.

Does GCE L4 LB replace original source IP with its own IP address?

[renaudguerin]

I've created L4 LBs in GCP using a "type: Loadbalancer" service definition, which doesn't create health checks for the nodes at the LB level (or didn't as of 1.4)

Am I correct in thinking that Kubelet's container level health checks don't help at all when a node is completely down (e.g. during a cluster version upgrade), and the L4 LB keeps sending traffic to it because it has no health checks to know it's down, and the node upgrade process hasn't temporarily removed the down node manually from the LB's backends either ?

This is obviously causing service disruptions and IIUC, the current solution to this is to use the source IP preservation feature, even when you don't care about the source IP, just because it has the side effect of creating proper health checks on L4 LBs ?

Correct ?

[thockin]

I reopened #14661

What you're doing is a viable workaround, but it is a workaround.

@mdelio

[renaudguerin]

@thockin That's what I thought yes, but there isn't currently a better solution apart from that SIPP side effect, correct ?

[thockin]

Correct

…

On Tue, Jan 10, 2017 at 11:29 AM, Renaud Guérin ***@***.***> wrote: @thockin <https://github.com/thockin> That's what I thought yes, but there isn't currently a better solution apart from that SIPP side effect, correct ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#32827 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFVgVGpA47_XKrifhCVjSlxqDO3DEe4Nks5rQ9wFgaJpZM4J-d_a> .