Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Database failover #203

Open
cirruspath opened this Issue · 19 comments

6 participants

@cirruspath

Hi. I have a master/slave setup already working with my redis database, and instead of the "round-robin" approach that is currently implemented in the config file, do you have any suggestions for implementing failover such that if the connection to host #1 fails, it moves to host #2? Same/similar configuration envisioned.

@michael-grunder

Hey,

Are you talking about the RedisArray functionality, or just generally.

The way I usually handle this sort of problem, is by wrapping my phpredis calls on the PHP side. I don't wrap calls to actual data functions themselves because that introduces too much of a performance penalty, but rather to the process of connecting.

We actually use a seperate Redis instance that containes our nodes, and then attempt to get a connection, failing over if one is down.

@cirruspath
@michael-grunder

I don't think phpredis supports this at the present moment. Right now I think if the connection times out it will just fail.

@nicolasff If you think failover is a reasonable idea, I can look into what it would take to make that work.

@nicolasff
Owner

@michael-grunder already looking into it, thanks!

@nicolasff
Owner

@cirruspath the way sessions work is by calling a specific read function in the extension when session_start() is called in PHP, and another write function at the end of the PHP script.
So implementing a failover for reads is pretty straightforward, but I'm not sure about the behaviour for the session write since it is very likely that a failed read from the master would be followed by a failed write at the end of the PHP script.

Should you not expect the write to fail in that case, and just let it fail? I assume you don't want to write the session to the slave, do you? That could lead to inconsistencies when the master comes back...

@michael-grunder any suggestions? My changes are on the session-failover branch.

@michael-grunder

@nicolasff I think you're probably correct to let the session handler fail on write. The only real way I can see around this would add quite a bit of complexity to phpredis (e.g. keep track of session server states, etc). I don't even know how we could do this properly, and if the use case was using an actual master->slave configuration, any attempt on our end to keep such context could get nuked when the master came back up.

Truth be told, however, I haven't ever used PHP sessions at all, so I'll have to dig into it a little deeper to come up with any good ideas. :)

@cirruspath
@nicolasff
Owner

@cirruspath yes, that's exactly what I did for the read path.
But at the end of a PHP script that uses sessions, the session data is written back to its data store which here would be a Redis write to the master. If we got the session data from the slave because the master was down, it is very likely that writing the data back would fail as well; if we write it to the slave we'll end up with inconsistencies. Does that make sense?

Consider the following script:

<?php

session_start(); /* this triggers a read on the master, which could fail and read from the slave instead. */

// do whatever with the session data

/** At the end of the script, the session data is written to Redis automatically,
  * whether $_SESSION has changed or not.
  * This will try to write to the master, and probably fail if the read had failed.
  * Writing the session to the slave leads to inconsistencies.
  */
?>

I'm keen on adding this feature, but I'm afraid it will still be kinda limited.

@cirruspath
@nicolasff nicolasff referenced this issue from a commit
@nicolasff nicolasff Write session data to master or slave (#203).
I had to introduce a "nothrow" flag in the RedisSock object in order
to avoid throwing exceptions in the context of a session write.
When PHP finishes and closes the session, it doesn't consider exceptions
to be valid and simply bails out without giving the caller a chance to
catch it. I replaced zend_throw_exception with redis_throw_exception which
checks for this flag and just returns if we are in a session close context.
e4ee953
@nicolasff
Owner

@cirruspath the changes are in, you can try out the feature using the "session-failover" branch on GitHub.
@michael-grunder does the exception change in e4ee953 look ok to you?

@cirruspath
@michael-grunder

@nicolasff Looks good to me. Similar to the destructor issue.

@snikkers

It works great, but if I understand correct . The "failover=" param only works with the short "host:port" , and not with the "tcp://" scheme ?

@snikkers

There only seems to be a issue when "auth=" is used, in that case : auth must be the same on both servers. And when the failover happends, the connection goes to the slave. But only the AUTH and GET are issued, and no new keys are SET.

At the same time the following error is logged :
[error] [client 127.0.0.1] PHP Fatal error: Uncaught exception 'RedisException' with message 'Connection closed' in [no active file]:0\nStack trace:\n#0 {main}\n thrown in [no active file] on line 0

@lightglitch

@nicolasff would this be available soon in a stable release?

@michael-grunder

Hey there,

There are a few feature branches that we want to get merged into master soon. I'm curious if you (or anyone else) has had success with the changes. @cirruspath, how did it work out for you?

@nicolasff You at all worried about these changes?

Cheers guys :)
Mike

@snikkers

Is it in a specific branch? I still have a test setup, so I can test it out .

@michael-grunder

I'm pretty sure it's in session-failover

Cheers,
Mike

@Ivun

Hello guys! Just reminding you about this feature. Any progress on that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.