Multiple Listener Addresses ?

[techmunk]

This is a feature request.
Would it be possible to add multiple listener addresses to DFP ? DFP could then get list of backends from multiple DFSL in different swarms.

Currently it's possible to run multiple DFSL in different swarms and have them update DFP and work correctly, until DFP restarts or scales. Adding multiple listener addresses to DFP could be a way around this.

I think this would also need an option added to DFSL to include a prefix to the service name to avoid collisions where two stacks have the same name in different swarms.

Thoughts ?

[thomasjpfan]

Can you clarify what you mean by "different swarms"?

[techmunk]

Actual different docker swarms on a separate physical cluster of machines.

Currently, you can for example run DFP on it's own swarm, a cluster of machines at the edge of the network for example.

Behind that, you can run multiple swarms across multiple racks (Or multiple Data Centers if you want).
A DFSL in each backend swarm can currently configure DFP to route requests to the appropriate swarm until it is restarted. At which point only one of the backend listeners can be queried. I propose adding the ability to query multiple backend listeners to auto configure DFP on startup with information from all provided listener addresses, assuming that address is accessible. (Should be ignored and check the next if it is not accessible).

Once you can query multiple listeners, the only other issue that arises is that of stacks in different swarms with the same name causing "collisions" in DFP. This can be a feature which could potentially be used for failover between swarms. i.e. two swarms with identical stack setups. One has a DFSL running, the other does not. Starting up DFSL in the second swarm, which then reconfigures DFP to route all services to the second swarm instead of the first.

Another scenario, is having DFP on the edge of the network, having a production swarm with DFSL, but then behind that, having development swarms (Again, on a different cluster of machines) that can also run DFSL and reconfigure the main DFP to route requests from the edge of the network to the development swarm. In this case, an option such as SERVICE_NAME_PREFIX on the DFSL could be used to help ensure no collisions occur with the result of running multiple DFSL's.

Does this make sense ? I can try and draw a diagram if not.

[techmunk]

I've drawn a diagram available here

I hope this helps.

[thomasjpfan]

I understand your use case. Both features are possible to include in DFP and DFSL.

1. Are you using com.df.outboundHostname to direct the edge DFP to the other swarms?

2. For the multiple listener addresses feature, I may need to add RELOAD_ATTEMPTS to control how many attempts DFP tries to query DFSL before moving on to the next one on the list. What do you think?

[techmunk]

@thomasjpfan exactly. I'm already using number 1, and it works wonderfully.
For number 2 I think that would make sense. I also think the timeout waiting should be reasonably short as well.

[thomasjpfan]

How are you notifying DFP that a cluster goes down? Are you forcing DFP to restart when a cluster goes down?

[techmunk]

We're using HAPRoxy healthcheck options (tcp-check etc..) and backup/failover options as described here and here which we've been able to setup by modifying the default DFP templates.

As far as I know DFP doesn't need to restart when a cluster goes down.

[thomasjpfan]

Multiple listener addresses has been added to dockerflow/docker-flow-proxy:18.07.03-68. You can set the number of reload attemps by setting the env var RELOAD_ATTEMPTS. The reload interval is set with the env var RELOAD_INTERVAL in ms.

DFSL has been updated with the DF_SERVICE_NAME_PREFIX feature added in dockerflow/docker-flow-swarm-listener:18.07.03-28. For example if DF_SERVICE_NAME_PREFIX=dev1, all services are prefixed with dev1_.

Let me know if this works for you!

[techmunk]

Thanks for your quick work @thomasjpfan This works exactly how I had hoped.