New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support redis failover in standalone #694

Open
afroDC opened this Issue Nov 16, 2017 · 6 comments

Comments

Projects
None yet
3 participants
@afroDC
Contributor

afroDC commented Nov 16, 2017

Enable the admin to add more than one redis-server to the configuration. oxAuth should handle switching to another cache if one is not available for connection.

@afroDC afroDC added the enhancement label Nov 16, 2017

@afroDC afroDC added this to the 3.2.0 milestone Nov 16, 2017

@yuriyz

This comment has been minimized.

Contributor

yuriyz commented Nov 25, 2017

Is it really about standalone or it is meant to support redis sentinel with automatic promotion of slave to master ?

https://redis.io/topics/sentinel

Sentinel requires minimum 3 servers for majority during voting.

@afroDC

This comment has been minimized.

Contributor

afroDC commented Nov 29, 2017

I'm not sure redis sentinel is what we want in this scenario, but if a user has multiple cache instances listed on Standalone, oxAuth should automatically switch to another instance if the first instance is non-responsive.

@yuriyz

This comment has been minimized.

Contributor

yuriyz commented May 25, 2018

@afroDC @isman @nynymike Standalone connection connects to one single redis server. If we add failover then it will be implemented programatically on oxcore project and means following:

  1. if connection is to server1 and server2 then oxcore will connect to server1 first.
  2. if server1 is unavailable then to server2.
  3. if oxauth is connected to server1 during some time and then crashed then it means that all redis data on server1 are lost and we are switching to server2. It at least means that:
  • all session data are lost and OP will start to send errors that sessions are invalid (from server1).
  • all users needs to authn&authz again.

If we are ok with above points then I can go ahead and implement failover with redis standalone connection.

@nynymike

This comment has been minimized.

Contributor

nynymike commented May 26, 2018

Let's hold on this. I thought we were using a redis proxy, so we don't need failover. Can't this be handled at the network level (i.e. use a virtal IP for the redis proxy which can route to one or more servers).

@afroDC

This comment has been minimized.

Contributor

afroDC commented May 30, 2018

@nynymike giving cache redundancy by having some sort of replication topology requires a smarter jedis client to handle one of the servers throwing an error and refusing connections. As it stands, twemproxy can't route traffic to a redis-cluster topology and using something like dynomitedb for replicating data instead doesn't give twemproxy the necessary signals it needs to know a redis-server is down.

We wouldn't need to use twemproxy if oxAuth could handle recognizing a server isn't responding and moving on to another in it's oxCacheConfiguration pool. For example, redis-cluster servers could be added to the configuration list. Or all dynomitedb proxies could be added. Any one that doesn't respond properly could be dropped for x amount of time.

There's currently not really any other option for giving a highly available cache system that I can find. The expectation seems to be that some client side error handling should be embedded.

@afroDC

This comment has been minimized.

Contributor

afroDC commented May 30, 2018

Alternatively, we could try to build something if we don't want to modify oxCore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment