Specify Marathon URL either via Master state or ZK connection string to support HA mode

[tobilg]

We're running Marathon in HA mode, meaning that we have 3 Marathon instances which are "clustered" via ZooKeeper.

So, for us it wouldn't necessarily make sense to use a static Marathon URL as it seems to be implemented now. If the exact instance would go down (for whatever reason), we could not make use of the HA feature of Marathon IMHO, and nixy would be in an inoperable state (if I understand correctly).

Possible solutions for (additionally) be able to use HA:

• Derive the actual webui_url of the Marathon framework via the Mesos Master's /state endpoint (I guess first you'd have to get the actual Master address via the ZK connection string of Mesos), or
• Use a ZK client to receive the leadership changes (not sure how this needs to be implemented though)

References:

• https://mesosphere.github.io/marathon/docs/high-availability.html

[martensson]

Thanks for the feedback and its a valid point you have there. Personally we dont have this problem since we run all our marathon nodes from our HA cluster behind an external Load Balancer. But I will in either case try to attend this issue.

I have on purpose tried to avoid adding a ZK client since it adds a lot to the complexity and Marathons REST API has everything that is needed. And to query Mesos could potentially add the same issue if that particular node is down.

An alternative solution would be to add a list of marathon endpoints, if one fails nixy would just retry with the next one in the list, and so on. Just have to see how to implement this the best way, but an idea would be a background health probe that checks all instances periodically.

[martensson]

Feature is now added in new release, went with adding a background endpoint probe.
Please make an issue if there is anything else 😄

https://github.com/martensson/nixy/releases/tag/v0.4.0

[tobilg]

Great 😀 That was fast!