Support client-side load balancing with the connection pool

[jrudolph]

How it could work:

• Allow customization of DNS resolution (see also Client API for manually resolving hostnames to IP addresses #92) which may return multiple addresses per host
• Allow overriding of target addresses when creating a connection
• In the pool, use different target addresses per slot
• Maybe a custom scheduling is also needed so that sequential requests don't all go the same then-idle connection

One way that users could hack in the first parts without changes to akka-http would be to use a custom ClientTransport returning a Flow that would transparently connect to different addresses each time it is materialized.

See also #1226

[jrudolph]

We already implemented max-connection-lifetime a long while ago to limit how long connections should stay alive until a new one is opened.

When #3202 is merged, you can customize resolution behavior for every new connection. I think that somewhat solves the first three points above:

• Allow customization of DNS resolution (see also Client API for manually resolving hostnames to IP addresses #92) which may return multiple addresses per host

• Allow overriding of target addresses when creating a connection

• In the pool, use different target addresses per slot

even though it doesn't given you a turnkey solution yet.

The remaining point would be to add documentation about this and:

• Maybe a custom scheduling is also needed so that sequential requests don't all go the same then-idle connection

I'm not sure that this is an actual problem because sequential requests are exactly that, sequential and the best way to use the pool is to run them all sequential over the same hot connection. If the sequential requests are only sequential by chance, then too much load on that one host should lead to the connection being less likely to be idle when the next request comes in, so requests should be spread out automatically.

[jrudolph]

It seems the essential functionality is there but could be explained better. I'll leave it open for that.

[jrudolph]

I created another ticket to discuss whether client-side load-balancing should be its own component. See #3505.