Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: predis multi slaves with a slave down #280

Closed
endymon opened this issue Sep 3, 2015 · 3 comments
Closed

Question: predis multi slaves with a slave down #280

endymon opened this issue Sep 3, 2015 · 3 comments
Labels
client-replication Replication (managed by Predis) feature redis-sentinel Replication (managed by redis-sentinel)
Milestone

Comments

@endymon
Copy link

endymon commented Sep 3, 2015

My question is, with a predis setup involving multiple configured slaves, If 1 or more slaves are down, how do you get predis to stop trying to send queries to the down slave(s)?

When I tested taking a slave down I got spammed by these:

Predis\\Connection\\ConnectionException: from 'EXISTS XXXXXX{THE_KEY}XXXXX' while connected to host: 'XXX{THE_HOST}XXX' MESSAGE: Connection refused [tcp://localhost:XX{PORT}XX]

And then after a short while the error message switched to:

'LOADING Redis is loading the dataset in memory'

These exceptions make sense if there is only one slave, but I would have assumed that in a multiple slave configuration predis either would by default (or by configuration parameter) fail-over silently to a NON down slave host. ONLY if all slaves were down could I see an exception propagating up to the client, but not in this case.

How does predis handle multiple slaves when 1 or more are failed or unavailable?
Is there such a configuration parameter?

Thanks
-- Matt

@xiconfjs
Copy link

Right now we are testing the replication-sentinel function of the predis 1.1-sentinel branch. And we are experiencing the same phenomenon. Is there an option to skip a slave which is re-syncing itself?

@nrk
Copy link
Contributor

nrk commented May 16, 2016

@endymon starting with Predis v1.1, trying to execute a read-only command on a unreachable slave will discard that slave and try again on the next one, or even the master if all of the slaves are offline (see the related changes). As for errors such as -LOADING they are not handled yet but I guess it should be fairly easy to intercept them to apply the same logic described above.

@xiconfjs the same applies to redis-sentinel, I will look into it.

@nrk nrk added redis-sentinel Replication (managed by redis-sentinel) feature client-replication Replication (managed by Predis) labels May 16, 2016
nrk added a commit that referenced this issue May 20, 2016
This prevents an early failure of the command execution on the client
when one slave gets back online but is still loading the dataset from
disk (when this happens, Redis returns the -LOADING error response).

This commit fixes #280.
@nrk
Copy link
Contributor

nrk commented May 20, 2016

Now Predis intercepts -LOADING responses, removes the "busy" slave from the pool and executes the command on the next slave (or master as last resort) instead of throwing an exception right away.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client-replication Replication (managed by Predis) feature redis-sentinel Replication (managed by redis-sentinel)
Development

No branches or pull requests

3 participants