-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Many redis connections #50
Comments
Not true. First of all, a large number of connections is not the reason behind database locking issues. It does simply tell you that you have a problem somewhere in your code and that your should handle it, maybe a concurrency issue. In general, when dealing with locking issues (no matter at which level), looking at the number of requests (connections) is useless. Your should investigate the matter of your requests and figure out WHAT they are doing. By the way, have you found any locks? I would be grateful if you could show me at least one issue in this regard. |
Well, if you are confident enough about your early statements, I highly encourage you to start your own project and propose it to the open source community, to be used in production-ready environments. I mean, go ahead! ) |
The number of connections for both the producer and the consumer is JUSTIFIED. Let me make it clear. By design, the RedisMQ is composed of many layers. Because we are talking about high performance (c), the main requirements for the requests to the Redis server of each layer are:
Considering the requirements, the best solution would be if each layer could operate on its own connections to Redis. Of course, new connections to Redis are made only when needed. For managing distributed locks, RedisSMQ uses the redlock package, which also maintains its own Redis connections (see https://www.npmjs.com/package/redlock). Here is the minimum number of connections needed for a producer :
The number and the purpose of the connections used by a consumer instance can be verified the same way. Consumers are more complex than producers, and include more layers (gc, heartbeat, etc.). RedisSMQ has been rigorously tested before being used in production at large scale for many projects that I developed. When benchmarking RedisSMQ, the numbers speak for themselves. |
@intech In anyway, thank you for your feedback |
Why create another product when it is possible to improve existing ones ;) |
You're welcome! If you find any issue please feel free to report it. |
If we are talking about high performance, then a large number of connections create locks at the redis level, and in our case, these are 15 connections per 1 Producer and 1 consumer:
Consumer - 11 connections
Producer - 4 connections
I researched the code and saw no reason to use that many connections.
For a producer, 1 connection is enough.
For the consumer, 3 connections are enough, one for the main process and two for the forks.
The text was updated successfully, but these errors were encountered: