Split-brain when redis master restarted without restart of its resec

[imcitius]

Hello

I found a reproducible situation: when you stop redis process on master, it becomes red in consul, but resec do not removes it from redis-master service.
Then after redis process is restarted, healthcheck sets it to green, and we now have double master setup, with slave connected to one of them.
Restart of resec process on former master resolves issue.

resec's log output from this master:

2018/03/19 17:05:42 [ERROR] Can't connect to redis running on 127.0.0.1:6379
2018/03/19 17:05:42 [INFO] Redis HealthCheck changed to NOT healthy
2018/03/19 17:05:42 [INFO] Received update for master from consul
2018/03/19 17:05:42 [INFO] No redis master services in Consul
2018/03/19 17:05:42 [INFO] Received update for master from consul
2018/03/19 17:05:42 [INFO] Redis master updated in Consul
2018/03/19 17:05:42 [INFO] Received update for master from consul
2018/03/19 17:05:42 [INFO] No redis master services in Consul
2018/03/19 17:05:47 [INFO] Received update for master from consul
2018/03/19 17:05:47 [ERROR] Can't connect to redis running on 127.0.0.1:6379
2018/03/19 17:05:52 [INFO] Redis HealthCheck changed to healthy
2018/03/19 17:05:52 [INFO] Trying to acquire leader lock
2018/03/19 17:05:52 [INFO] Received update for master from consul
2018/03/19 17:05:52 [ERROR] Found more than one master registered in Consul

[burdandrei]

@imcitius just saw it. Will check, Thanks!