Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection error not thrown when Sentinel master name is wrong #397

Closed
jehy opened this issue Nov 16, 2016 · 10 comments
Closed

Connection error not thrown when Sentinel master name is wrong #397

jehy opened this issue Nov 16, 2016 · 10 comments
Labels

Comments

@jehy
Copy link

jehy commented Nov 16, 2016

Hi! I happened to have a wrong master name in my config. ioredis kept reconnecting and didn't throw error event. If I enable debug mode, I can see error:

 ioredis:redis write command[0] -> sentinel(sentinels,redis-beta1111) +0ms
  ioredis:SentinelConnector failed to connect to sentinel xxx.xxx.xxx.xxx:yyy because ReplyError: ERR No such master with that name +1ms
  ioredis:SentinelConnector All sentinels are unreachable. Retrying from scratch after 122 +0ms

But how can I catch it in my application?

P.S. using ioredis v 2.4.0

@luin
Copy link
Collaborator

luin commented Nov 16, 2016

Check out sentinelRetryStrategy option.

@luin luin added the question label Nov 16, 2016
@jehy
Copy link
Author

jehy commented Nov 16, 2016

@luin yeah, I can see retries happening - but how can I get error message?
Will it be in error event if I return non-number message from sentinelRetryStrategy?

@luin
Copy link
Collaborator

luin commented Nov 16, 2016

Yes, an error event will be emitted when sentinelRetryStrategy returns a non-number. Besides, Redis#connect() will be rejected with the same error.

@jehy
Copy link
Author

jehy commented Nov 16, 2016

Hmm, error message is error event is much less detailed, it only says Error: All sentinels are unreachable. Last error: Connection is closed.
Message No such master with that name would be much better. But that's at least something. Thanks!

@luin
Copy link
Collaborator

luin commented Nov 16, 2016

That's strange. How many sentinels do you have? Given ioredis only emits the last error (https://github.com/luin/ioredis/blob/5f740758593bb0e2342d25c1f84e43418d6cb868/lib/connectors/sentinel_connector.js#L78-L79), it seems ioredis tried to connect to another sentinel and got Connection is closed after ERR No such master with that name.

@jehy
Copy link
Author

jehy commented Nov 16, 2016

I have the following config:

"redis": { "sentinels": [ { "host": "localhost", "port": 26379 }, { "host": "localhost", "port": 26380 } ], "name": "mymaster", "keyPrefix": "kbk:", "enableOfflineQueue": false },

And I think that error message should be the same for both sentinels since name is wrong.

@luin
Copy link
Collaborator

luin commented Nov 16, 2016

Could you post full logs here including retries for the two sentinels?

@jehy
Copy link
Author

jehy commented Nov 16, 2016

Yeah, here is the last retry attempt (99 of 100):

ioredis:SentinelConnector All sentinels are unreachable. Retrying from scratch after 99 +1ms
ioredis:redis status[127.0.0.1:26379]: ready -> close +2ms
ioredis:connection skip reconnecting since the connection is manually closed. +1ms
ioredis:redis status[127.0.0.1:26379]: close -> end +0ms
ioredis:redis status[localhost:26379]: [empty] -> connecting +95ms
ioredis:redis queue command[0] -> sentinel(get-master-addr-by-name,mymaster1) +1ms
ioredis:redis status[127.0.0.1:26379]: connecting -> connect +1ms
ioredis:redis status[127.0.0.1:26379]: connect -> ready +0ms
ioredis:connection send 1 commands in offline queue +0ms
ioredis:redis write command[0] -> sentinel(get-master-addr-by-name,mymaster1) +0ms
ioredis:redis write command[0] -> sentinel(sentinels,mymaster1) +2ms
ioredis:SentinelConnector failed to connect to sentinel localhost:26379 because ReplyError: ERR No such master with that name +4ms
ioredis:redis status[localhost:26380]: [empty] -> connecting +1ms
ioredis:redis queue command[0] -> sentinel(get-master-addr-by-name,mymaster1) +0ms
ioredis:connection error: Error: connect ECONNREFUSED 127.0.0.1:26380 +1ms
ioredis:redis status[localhost:26380]: connecting -> close +0ms
ioredis:connection skip reconnecting because retryStrategy is not a function +1ms
ioredis:redis status[localhost:26380]: close -> end +0ms
ioredis:SentinelConnector All sentinels are unreachable. Retrying from scratch after 100 +1ms
ioredis:redis status[127.0.0.1:26379]: ready -> close +2ms
ioredis:connection skip reconnecting since the connection is manually closed. +0ms
ioredis:redis status[127.0.0.1:26379]: close -> end +0ms
ioredis:redis status[localhost:26379]: [empty] -> connecting +99ms
ioredis:redis queue command[0] -> sentinel(get-master-addr-by-name,mymaster1) +0ms
ioredis:redis status[127.0.0.1:26379]: connecting -> connect +1ms
ioredis:redis status[127.0.0.1:26379]: connect -> ready +1ms
ioredis:connection send 1 commands in offline queue +0ms
ioredis:redis write command[0] -> sentinel(get-master-addr-by-name,mymaster1) +0ms
ioredis:redis write command[0] -> sentinel(sentinels,mymaster1) +1ms
ioredis:SentinelConnector failed to connect to sentinel localhost:26379 because ReplyError: ERR No such master with that name +2ms
ioredis:redis status[localhost:26380]: [empty] -> connecting +0ms
ioredis:redis queue command[0] -> sentinel(get-master-addr-by-name,mymaster1) +0ms
ioredis:connection error: Error: connect ECONNREFUSED 127.0.0.1:26380 +2ms
ioredis:redis status[localhost:26380]: connecting -> close +0ms
ioredis:connection skip reconnecting because retryStrategy is not a function +0ms
ioredis:redis status[localhost:26380]: close -> end +0ms
ioredis:SentinelConnector failed to connect to sentinel localhost:26380 because Error: Connection is closed. +1ms
ioredis:SentinelConnector All sentinels are unreachable and retry is disabled, emitting error...

@luin
Copy link
Collaborator

luin commented Nov 16, 2016

It shows 127.0.0.1:26380 refuse the connection ECONNREFUSED instead of returning ERR No such master with that name.

@jehy
Copy link
Author

jehy commented Nov 16, 2016

Oops. My bad. Sentinel really listened only on 26379. Stupid mistake, sorry for taking your time. Thank you very much!

@jehy jehy closed this as completed Nov 16, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants