Cannot connect to password-authenticated sentinels

[Yenya]

Hello!

I am trying to connect to a set of password-authenticated sentinels using the following connect string:

my $r = Mojo::Redis->new(
        'redis://:somepass@redis_instance?sentinel=10.0.0.1:6381&sentinel=10.0.0.2:6381'
);

Accordng to strace, Mojo::Redis connects and authenticates itself to the sentinel, but then gets stuck probably waiting for more data, even though according to strace the respolse is complete:

getpeername(3, {sa_family=AF_INET, sin_port=htons(6381), sin_addr=inet_addr("10.0.0.1")}, [256->16]) = 0
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
poll([{fd=3, events=POLLIN|POLLPRI|POLLOUT}], 1, 501) = 1 ([{fd=3, revents=POLLOUT}])
write(3, "*2\r\n$4\r\nAUTH\r\n$8\r\nsomepass\r\n*3\r\n$8\r\nSENTINEL\r\n$23\r\nget-master-addr-by-name\r\n$14\r\nredis_instance\r\n", 123) = 123
poll([{fd=3, events=POLLIN|POLLPRI}], 1, 501) = 1 ([{fd=3, revents=POLLIN}])
brk(NULL)                               = 0x2984000
brk(0x29c0000)                          = 0x29c0000
read(3, "+OK\r\n*2\r\n$8\r\n10.0.0.1\r\n$4\r\n6380\r\n", 131072) = 39
poll([{fd=3, events=POLLIN|POLLPRI}], 1, 501) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLPRI}], 1, 501) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLPRI}], 1, 501) = 0 (Timeout)

(addresses and passwords edited, so the actual bytes read do not match). Could it be that Mojo::Redis expects some more data from sentinel?

The following code using Redis.pm works:

my $r = Redis->new(
        sentinels => [ '10.0.0.1:6381', '10.0.0.2:6381', ],
        sentinels_password => 'somepass',
        password => 'somepass',
        service => 'redis_instance',
) or die;

Thanks!

-Yenya

[jhthorsen]

I'm not sure. I don't have sentinels set up myself, so not sure how to debug the issue. But one thing you could do is to set the MOJO_REDIS_DEBUG environment variable to 1 instead of using strace. I think that will make it easier to understand what is going on.

[Yenya]

OK, here you are:

[8cab1d17f546de3985bc2bf9561ee16f] SENTINEL DISCOVERY redis://xxx.xxx.xx.xx:6381?sentinel=xxx.xxx.xx.xy%3A6381&sentinel=xxx.xxx.xx.xx%3A6381 (blocking=1)
[8cab1d17f546de3985bc2bf9561ee16f] <<< (*2\r\n$4\r\nAUTH\r\n$42\r\n(password edited)\n)
[8cab1d17f546de3985bc2bf9561ee16f] <<< (*3\r\n$8\r\nSENTINEL\r\n$23\r\nget-master-addr-by-name\r\n$5\r\ncache\r\n)
[8cab1d17f546de3985bc2bf9561ee16f] >>> (+OK\r\n*2\r\n$13\r\nxxx.xxx.xx.xx\r\n$4\r\n6380\r\n)

[Yenya]

If you want to reproduce this yourself, a minimal one-server one-sentinel configuration is the following:

$ cat redis.conf
port 7000
requirepass S3cr3t

$ cat sentinel.conf
port 7001
sentinel monitor mymaster 127.0.0.1 7000 1
sentinel down-after-milliseconds mymaster 60000
sentinel auth-pass mymaster S3cr3t
requirepass "S3cr3t"

$ redis-server redis.conf &
$ redis-sentinel sentinel.conf &
$ perl -MRedis -E 'my $r = Redis->new(service=>"mymaster", password=>"S3cr3t", sentinels=>["127.0.0.1:7001"], sentinels_password=>"S3cr3t"); $r->set("/test" => 42, EX => 100); say $r->get("/test")'
42

$ MOJO_REDIS_DEBUG=1 perl -MMojo::Redis -E 'my $r = Mojo::Redis->new("redis://:S3cr3t\@mymaster?sentinel=127.0.0.1:7001"); $r->db->set("/test" => 42); say $r->db->get("/test");'
[6caf1adfbb08afcb4453a3a76624da93] SENTINEL DISCOVERY redis://127.0.0.1:7001?sentinel=127.0.0.1%3A7001 (blocking=1)
[6caf1adfbb08afcb4453a3a76624da93] <<< (*2\r\n$4\r\nAUTH\r\n$6\r\nS3cr3t\r\n)
[6caf1adfbb08afcb4453a3a76624da93] <<< (*3\r\n$8\r\nSENTINEL\r\n$23\r\nget-master-addr-by-name\r\n$8\r\nmymaster\r\n)
[6caf1adfbb08afcb4453a3a76624da93] >>> (+OK\r\n*2\r\n$9\r\n127.0.0.1\r\n$4\r\n7000\r\n)
[... it hangs there forever ...]

Let me know if you need anything more to reproduce this.

[Yenya]

@jhthorsen: did you have time to look at this? Thanks!