Handling multiple redis endpoints without Sentinel

[twexler]

As part of #1554, I added redis sentinel support, but there is an outstanding TODO with the standard redis implementation to accept multiple endpoints.

On the tip of it, it looks like ClusterClient which uses the Cluster Client spec may be the right way to go, but I'm concerned that was not necessarily the intention for being able to specify multiple endpoints.

The only other solution I can think of is having Athens be able to connect to all specified servers and run an individual lock on all of the servers in non-clustering mode. This seems a bit overkill, but would satisfy the need for a basic lock mechanism with multiple endpoints.

Thoughts? cc @marwan-at-work

[twexler]

For reference the TODO is here: https://github.com/gomods/athens/blob/master/config.dev.toml#L288L296

[marwan-at-work]

Thanks for opening up the discussion, I'm a bit behind on new Redis features like Sentinel and clustering...I'd love to take the time to do it, maybe bringing back our zoom office hours and having you @twexler give us a walk through would be amazing 😄

But just for a bit of history: I implemented the Redis SingleFlight thinking it's gonna be totally okay to have a single instance for two reasons:

1. If one wants a distributed lock we already have two implementations: Etcd and GCS
2. Athens is mainly meant for internal usage, so I can't imagine a company so big that needs a distributed lock just for their internal usage :)

So my question is:

Is Redis Sentinel truly a whole new system that warrants us to expose two different Redis configurations?

[arschles]

Is Redis Sentinel truly a whole new system that warrants us to expose two different Redis configurations?

I think if we do add sentinel, we should expose it as a new config section, because it's fundamentally different.

I'm not sure if we should support sentinel for locks at all though. Do we know if someone will use it? @twexler maybe you know?

[twexler]

I'm not sure if we should support sentinel for locks at all though.

Could you expand on this?

Do we know if someone will use it? @twexler maybe you know?

This was partially self-serving ;)

Other than that, I have seen other uses of it...but I'm not actively aware of who is using it.

[arschles]

@twexler if you're using sentinel, that counts! I just want to know that this has a chance of being used, before we make a change to our configs