Automatic redis monitoring and failover based on Go
Switch branches/tags
Nothing to show
Clone or download


Automatic redis monitoring and failover.

Why not redis-sentinel?

redis-sentinel is very powerful, why not use it but build a new one?

  1. I want to support not only redis, but also LedisDB's monitoring and failover.
  2. I want to embed it into another service like xcodis.
  3. I just want to learn how to build a distributed service using raft. :-)

Install and usage

go get

First you must start redis and build the replication topology by yourself. e.g, is master and is slave.

Use without raft, only single node

redis-failover -addr= -masters=

addr is redis-failover HTTP listen address, masters is the master redis server you want to monitor.

you can use a config file too, like redis-failover -config=./etc/failover.toml. see failover.toml.

You can add master dynamically from HTTP, using httpie below:

http POST :11000/master masters==

Use raft, with only single node

redis-failover -addr= -masters= -raft_addr= -raft_data_dir=./var0 -raft_cluster= -broker=raft

raft_addr is the raft listen address for inner raft communication. raft_data_dir is the store path for raft, raft_cluster is the raft cluster, here only one node.

broker is the cluster type, now "raft" or "zk".

You must know that if you want to use raft to avoid redis-failover single point of failure, you should not use only one raft node in production.

Use raft, with multi nodes

redis-failover -addr= -masters= -raft_addr= -raft_data_dir=./var0 -raft_cluster=,, -broker=raft

redis-failover -addr= -masters= -raft_addr= -raft_data_dir=./var1 -raft_cluster=,, -broker=raft

redis-failover -addr= -masters= -raft_addr= -raft_data_dir=./var2 -raft_cluster=,, -broker=raft

raft_cluster now contains three raft nodes, so if one node down, other two can still work correctly.

Use zookeeper, with only single node

redis-failover -addr= -masters= -zk_addr= -zk_path=/zk/redis/failover -broker=zk

zk_addr is the remote zookeeper address, multi zk is seperated by comma.

zk_path is the zookeeper base directory you want to save your data, the prefix must be "/zk".

Use zookeeper, with multi nodes

redis-failover -addr= -masters= -zk_addr= -zk_path=/zk/redis/failover -broker=zk

redis-failover -addr= -masters= -zk_addr= -zk_path=/zk/redis/failover -broker=zk

redis-failover -addr= -masters= -zk_addr= -zk_path=/zk/redis/failover -broker=zk


After you start redis-failover and set master redis, redis-failover will check it automatically. After it finds the master is down, it will do failover, the failover step is:

  1. Elect a slave which has the most up-to-date data with master to the candidate, use INFO REPLICATION to check.
  2. Promote the candidate to the master, use SLAVEOF NO ONE.
  3. Let other slaves replicate from the new master, use SLAVEOF new_master_host new_master_port.

redis-failover will log some messages for failover, like:

[2015/02/10 14:10:18] group.go:64 [Error] do ROLE command for error: EOF, try again
[2015/02/10 14:10:18] group.go:56 [Error] dial error: dial tcp connection refused, try again
[2015/02/10 14:10:18] group.go:56 [Error] dial error: dial tcp connection refused, try again
[2015/02/10 14:10:18] app.go:166 [Error] check master err Node is down, do failover
[2015/02/10 14:10:18] group.go:259 [Info] select slave as new master, priority:100, repl_offset:29

If the failover failed, redis-failover will stop to check this redis to avoid future unexpected errors, so at that time, you may fix it manually by yourself.


  • Redis version >= 2.8.12, redis-failover will use redis ROLE command to fetch the replication topology from master.