Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request queue size is not cleared on reconnect #1843

Closed
Hixon10 opened this issue Sep 4, 2021 · 4 comments
Closed

Request queue size is not cleared on reconnect #1843

Hixon10 opened this issue Sep 4, 2021 · 4 comments
Labels
status: waiting-for-feedback We need additional information before we can continue

Comments

@Hixon10
Copy link

Hixon10 commented Sep 4, 2021

Bug Report

  1. There is a Java (spring) service, which uses Redis cluster with help lettuce.
  2. We have a setting spring.redis.lettuce.pool.maxActive == 100_000
  3. At some moment, the service lose network connectivity to one of redis cluster master nodes.
  4. The service has a bunch of background queries, which are executed periodically (e.g. we execute ping query each second).
  5. After some time, io.lettuce.core.protocol.DefaultEndpoint#QUEUE_SIZE is equal to 100_000 for some channels
  6. As result, the service cannot execute any queries, using this connection. We got an error, like Request queue size exceeded: 100000. Commands are not accepted until the queue size drops.
  7. After some time, the network connective issue is resolved.
  8. The problem is, that the application still has io.lettuce.core.protocol.DefaultEndpoint#QUEUE_SIZE = 100_000. If I manually set this field to 0, the application starts working.

Environment

  • Lettuce version(s): 6.1.2.RELEASE
  • Redis version: 6.2.5

Maybe, it is similar to #616

@mp911de
Copy link
Collaborator

mp911de commented Sep 8, 2021

Lettuce auto-reconnects and replays commands upon reconnect and that should drain the queue.

@tishun
Copy link
Collaborator

tishun commented Jun 12, 2024

@Hixon10 are you able to provide a reproducible example where this problem can be examined?

@tishun tishun added the status: waiting-for-feedback We need additional information before we can continue label Jun 12, 2024
@Hixon10
Copy link
Author

Hixon10 commented Jun 12, 2024

@tishun unfortunately, no. It was long time ago, I don't have any context for this issue anymore.

@tishun
Copy link
Collaborator

tishun commented Jun 15, 2024

@tishun unfortunately, no. It was long time ago, I don't have any context for this issue anymore.

Thank you. I will be closing this for now and if somebody else runs into this again we can reopen.

@tishun tishun closed this as completed Jun 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: waiting-for-feedback We need additional information before we can continue
Projects
None yet
Development

No branches or pull requests

3 participants