sync-catalog: Support ClusterIP endpoints

[mitchellh]

Context: hashicorp/consul#4710 (comment)

We should support ClusterIP by using endpoints since we already assume that pods are routable given how we treat NodePort.

This is a good first issue because we already have the machinery to sync endpoints (for NodePort) and we already have test examples showing how this would work, so this is a great way to dive in and enhance the functionality.

/cc @roboll

[jackpizarro]

This would be great to have. In a setup where the HAProxy farm runs itself on pods and can reach any endpoint ClusterIP would be ideal to expose services.

Kudos for the work done so far, very useful!

[civik]

Just curious why you assume the pods are routable? Typically pods are only routable when using a cloud providers CNI plugins. Almost anything on-prem, most commonly Flannel, will have non-routable pod IPs.

I'm assuming this is why when I try to deploy the client only, the pod goes into CLBO with the log message [WARN ] to-consul/sink: error querying services, will retry: err="Get http://<host ip>:8500/v1/catalog/services?index=1&stale=&wait=60000ms: dial tcp <host ip>:8500: connect: connection refused"

[mitchellh]

Just curious why you assume the pods are routable? Typically pods are only routable when using a cloud providers CNI plugins. Almost anything on-prem, most commonly Flannel, will have non-routable pod IPs.

There are two reasons!

First, with any new release ("0.1"-type software), we focus on getting started experience as a primary use case. Most getting started experiences with K8S are either minikube or a hosted solution. All hosted solutions have easily routable pod IPs as you stated.

Second, we also of course survey our customers (which are generally enterprises, global 2000 size) and get their use case so that we're designing with that in mind even if we don't solve their problem day 1. A surprisingly large number have routable pod IPs!

Given both of these reasons, it felt like a good starting point. The architecture we built does not limit us to this, however.

I'm assuming this is why when I try to deploy the client only, the pod goes into CLBO with the log message [WARN ] to-consul/sink: error querying services, will retry: err="Get http://<host ip>:8500/v1/catalog/services?index=1&stale=&wait=60000ms: dial tcp <host ip>:8500: connect: connection refused"

That's not trying to route to pods at all, its trying to route to the host. Our default consul helm chart will deploy clients using hostPort to expose on the host IP. You can easily override this though, its just a default we use since thats what our helm chart does.

If there are any configuration options you'd want opened up to your use case, please let us know and we'll take a look!

[kuroneko25]

This is going to be a big improvement for us as well. We have flat routable pod IPs and would like to avoid using NodePort Services. When can we expect this feature go live?

[adilyse]

Support for optionally syncing ClusterIP services has been merged and will be released shortly!