Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Call bind(2) before connect(2) if exists "bind ADDR" in redis.conf #1155

Closed
wants to merge 1 commit into from

3 participants

hirose31 Z. Liu Matt Stancliff
hirose31

Redis master recognizes slave's IP address by getpeername(2) so that address becomes an address assigned slave's eth0 even if IP aliased (eth0:1) in slave linux box and bind <IP address assigned eth0:1) in redis.conf.

This is not a promblem in simple master/slave replication except that redis-clit -h MASTER info | grep slave shows wrong slave's IP address.

Now I introduced Redis Sentinel. Sentinel gets slaves address from result of info command to master, so sentinel recognizes not slave's eth0:1 but slave's eth0 as slave's IP address.

Sentinel's health check to slave will fail because slave only listen eth0:1. If redis master crashed, sentinel cannot failover because there is no healthy slave.

To fix it, slave calls bind(2) before connect(2) to master if there is bind <address> in redis.conf and that address is not local address (127.0.0.0/8).

Could you give me any advice?

Redis master:
  IP address:
    eth0:   10.0.0.100

Redis slave:
  IP address:
    eth0:   10.0.0.200
    eth0:1: 10.0.0.201
  redis.conf:
    bind 10.0.0.201

Redis sentinel:
  IP address:
    eth0:   10.0.0.50
  sentinel.conf:
    sentinel monitor mymaster 10.0.0.100 6379 1

$ redis-cli -h 10.0.0.201 slaveof 10.0.0.100 6379

$ redis-cli -h 10.0.0.100 info | grep slave
slave0:10.0.0.200,6379,online  # not 10.0.0.201

$ redis-cli -h 10.0.0.50 sentinel slaves mymaster | grep -A1 ip
ip
10.0.0.200  # not 10.0.0.201

hirose31 hirose31 Call bind(2) before connect(2) if exists "bind ADDR" in redis.conf
Redis master recognizes slave's IP address by getpeername(2) so that address becomes an address assigned slave's eth0 even if IP aliased (eth0:1) in slave linux box and ```bind <IP address assigned eth0:1)``` in redis.conf.

This is not a promblem in simple master/slave replication except that ```redis-clit -h MASTER info | grep slave``` shows wrong slave's IP address.

Now I introduced Redis Sentinel. Sentinel gets slaves address from result of ```info``` command to master, so sentinel recognizes not slave's eth0:1 but slave's eth0 as slave's IP address.

Sentinel's health check to slave will fail because salve only listen eth0:1. If redis master crashed, sentinel cannot failover because there is no healthy slave.

To fix it, slave calls bind(2) before connect(2) to master if there is ```bind <address>``` in redis.conf and that address is not local address (127.0.0.0/8).
d6b4698
hirose31

@antirez
I think this issue is a critical problem when running some redis-servers on same port(6379) on same server.
What do you think about?

Z. Liu

+1

This is very helpful. In out case, we running redis as slave on a pc, the master is a remote server which connected by vpn. For redundancy reason, there are multiple vpn connection in use, the route is decided by route policy. Then, the problem comes, if the active vpn is broken, redis will have to reconnect to the master since the address used to connect master will change. Before redis 2.8, this mean a full data re-transfer. If we can bind the client on a fixed local address, from the TCP's side, redis connection will be transparent to the vpn things. Thanks.

Matt Stancliff
Collaborator

This has been fixed elsewhere in #2110

Matt Stancliff mattsta closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 14, 2013
  1. hirose31

    Call bind(2) before connect(2) if exists "bind ADDR" in redis.conf

    hirose31 authored
    Redis master recognizes slave's IP address by getpeername(2) so that address becomes an address assigned slave's eth0 even if IP aliased (eth0:1) in slave linux box and ```bind <IP address assigned eth0:1)``` in redis.conf.
    
    This is not a promblem in simple master/slave replication except that ```redis-clit -h MASTER info | grep slave``` shows wrong slave's IP address.
    
    Now I introduced Redis Sentinel. Sentinel gets slaves address from result of ```info``` command to master, so sentinel recognizes not slave's eth0:1 but slave's eth0 as slave's IP address.
    
    Sentinel's health check to slave will fail because salve only listen eth0:1. If redis master crashed, sentinel cannot failover because there is no healthy slave.
    
    To fix it, slave calls bind(2) before connect(2) to master if there is ```bind <address>``` in redis.conf and that address is not local address (127.0.0.0/8).
This page is out of date. Refresh to see the latest.
Showing with 16 additions and 8 deletions.
  1. +13 −5 src/anet.c
  2. +1 −1  src/anet.h
  3. +1 −1  src/migrate.c
  4. +1 −1  src/replication.c
18 src/anet.c
View
@@ -200,10 +200,10 @@ static int anetCreateSocket(char *err, int domain) {
#define ANET_CONNECT_NONE 0
#define ANET_CONNECT_NONBLOCK 1
-static int anetTcpGenericConnect(char *err, char *addr, int port, int flags)
+static int anetTcpGenericConnect(char *err, char *addr, int port, int flags, char *bindaddr)
{
int s;
- struct sockaddr_in sa;
+ struct sockaddr_in sa, src_sa;
if ((s = anetCreateSocket(err,AF_INET)) == ANET_ERR)
return ANET_ERR;
@@ -225,6 +225,14 @@ static int anetTcpGenericConnect(char *err, char *addr, int port, int flags)
if (anetNonBlock(err,s) != ANET_OK)
return ANET_ERR;
}
+
+ if (bindaddr && strncmp(bindaddr, "127.", 4) != 0) {
+ memset(&src_sa, 0, sizeof(src_sa));
+ src_sa.sin_family = AF_INET;
+ src_sa.sin_addr.s_addr = inet_addr(bindaddr);
+ bind(s, (struct sockaddr *)&src_sa, sizeof(src_sa));
+ }
+
if (connect(s, (struct sockaddr*)&sa, sizeof(sa)) == -1) {
if (errno == EINPROGRESS &&
flags & ANET_CONNECT_NONBLOCK)
@@ -239,12 +247,12 @@ static int anetTcpGenericConnect(char *err, char *addr, int port, int flags)
int anetTcpConnect(char *err, char *addr, int port)
{
- return anetTcpGenericConnect(err,addr,port,ANET_CONNECT_NONE);
+ return anetTcpGenericConnect(err,addr,port,ANET_CONNECT_NONE,NULL);
}
-int anetTcpNonBlockConnect(char *err, char *addr, int port)
+int anetTcpNonBlockConnect(char *err, char *addr, int port, char *bindaddr)
{
- return anetTcpGenericConnect(err,addr,port,ANET_CONNECT_NONBLOCK);
+ return anetTcpGenericConnect(err,addr,port,ANET_CONNECT_NONBLOCK,bindaddr);
}
int anetUnixGenericConnect(char *err, char *path, int flags)
2  src/anet.h
View
@@ -40,7 +40,7 @@
#endif
int anetTcpConnect(char *err, char *addr, int port);
-int anetTcpNonBlockConnect(char *err, char *addr, int port);
+int anetTcpNonBlockConnect(char *err, char *addr, int port, char *bindaddr);
int anetUnixConnect(char *err, char *path);
int anetUnixNonBlockConnect(char *err, char *path);
int anetRead(int fd, char *buf, int count);
2  src/migrate.c
View
@@ -151,7 +151,7 @@ void migrateCommand(redisClient *c) {
/* Connect */
fd = anetTcpNonBlockConnect(server.neterr,c->argv[1]->ptr,
- atoi(c->argv[2]->ptr));
+ atoi(c->argv[2]->ptr), server.bindaddr);
if (fd == -1) {
addReplyErrorFormat(c,"Can't connect to target node: %s",
server.neterr);
2  src/replication.c
View
@@ -670,7 +670,7 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
int connectWithMaster(void) {
int fd;
- fd = anetTcpNonBlockConnect(NULL,server.masterhost,server.masterport);
+ fd = anetTcpNonBlockConnect(NULL,server.masterhost,server.masterport,server.bindaddr);
if (fd == -1) {
redisLog(REDIS_WARNING,"Unable to connect to MASTER: %s",
strerror(errno));
Something went wrong with that request. Please try again.