Multiple apiservers fighting over endpoint

[philk]

I've done a bunch of searching for an answer to this but don't see a real solution. If you're running with the currently recommended solution for multiple api servers they continuously update the kubernetes service endpoint. This doesn't really cause any problems until you try to use the Ingress resource since the endpoint modification causes the ingress controller (at least the haproxy one, I don't see any reason why the same wouldn't be true of any other) to continuously restart.

It also spams the kube-proxy logs with:

kube-proxy[1580]: I1210 21:44:08.572592    1580 proxier.go:352] Setting endpoints for "default/kubernetes:" to [10.0.167.163:443]

every couple seconds.

It looks like #8649 is the latest attempted fix, however (at least on AWS) there's no address to advertise since ELBs aren't static IPs. #6672 references #6995 but neither of those end up in real solutions. #7273 looks like it was exactly what I'm looking for but then #7704 seems to have removed it without any explanation that I saw.

This is all a really long winded way to say, what's the real recommended way to run multiple API servers?

[j3ffml]

cc @mikedanese

[mikedanese]

This would be fixed by setting master count to the number of apiservers. Currently it's always set to one. @lavalamp should we expose that as a command line flag or were you waiting for a better way to derive that number?

[mikedanese]

#18628 will fix this. We can come up with a fancy way to derive that number later and deprecate that configuration.

[lavalamp]

@mikedanese Just to confirm, yes, exposing it as a flag is the right thing. Thanks.

[mikedanese]

No. This will be in v1.2