Client-side load balancing

[peterbourgon]

• Manual service discovery (list of endpoints)
• DNS-based service discovery (single IP and SRV)
• Consul
• etcd

/cc @dahernan can you drop your use-case in here? It would be very helpful!

[dahernan]

Use case:

We currently have a microservice to do service discovery (based in etcd), this service gets the list of server:port to connect, and also notify any changes with a channel.

Current solution:

The current solution to connect microservices is put HAProxy in the middle of services to do load balance, and add/remove services, the config for HAProxy is feed with the list of services from service discovery.

Pros:

• HAProxy rocks. Good, stable loadbalancer and easy to monitor.
• The client of the microservice, connects only to one point

Cons:

• HAProxy is another piece in the middle, it could increase the latency, and is more complex to deploy
• For a client is two levels of service discovery, you have to discovery the HAProxy instead of the end machines, it adds more complexity to service discovery

Desire Solution:

Be able to use a http.Transport for the http client that supports service discovery and load balancing, so it removes HAProxy as a middleware. (more or less what Netflix Ribbon[1] does).

Pros:

• Simple model to deploy
• No need to do service discovery 'twice'
• it could be easily integrated with Circuit Breakers

Cons:

• Replace some of the features and monitoring from HAProxy could be really hard an error prone.
• HAProxy allows dummy clients in multiple languages (they don't need to know about load balancing and service discovery)

[1]https://github.com/Netflix/ribbon

[peterbourgon]

Brilliant, thank you! Can you say more about how you interact with etcd?

[dahernan]

Every micro-service emits heartbeats that are register in a keyspace:
/heartbeats/<service-name>/<host-name>/<port> : timestamp

Discovery service polls that keyspace and maintain another keyspace with the list of the available services based in the last activity

You only need to call Discovery service to get the list of host given a service name.

[peterbourgon]

Given #67 I think the way this would work would be implementing an etcd Publisher that respected endpoint healthiness. Then that publisher can feed an e.g. RoundRobin LoadBalancer. And then, building an http.Transport that wrapped that load-balanced endpoint.

Does that make sense to you? Any obvious impedance mismatches?

[dahernan]

Yeah, that sounds good to me.
I don't see any issue at this moment.

[peterbourgon]

So, I'll close this in favor of #68. Thanks for the feedback!