How can I preheat ConnectionPool?

[ethe]

ConnectionPool's initializer does not create connections immediately, in some cases this strategy causes some problem, so is there anyway to preheat connection pool (create some IDLE connections in pool initializing)?

[andymccurdy]

Short answer, there's no automatic way to do this. You could do this manually by acquiring N number of connections, running a common on each of them, then releasing them all back to the connection pool.

[tzickel]

What problems does it cause ?

[ethe]

What problems does it cause ?

At the beginning of the server starting, there is no connections to Redis, and tons of requests make service snow crash.

[tzickel]

Not sure how it's related to your problem ? If you preheat tons of connections at start won't that just make it crash sooner ?

[ethe]

Not sure how it's related to your problem ? If you preheat tons of connections at start won't that just make it crash sooner ?

The problem is, if there is no connection at the beginning, server's performance decreased heavily (cause connection pool should make connection before every request, it is a heavy overhead). And, if service accepts requests' count as same as usual, there are lots requests hang up, this strategy would makes more abnormal request (users would retry and retry if they find that their request slower than before, it is a common case on web server). Then, we get a snow crash.

[ethe]

Lots of Redis driver accepts optional max_idle_connections param let user choose whether keep connections in idle time (includes beginning). Yeah there is a manual way to do this in redis-py (call make_connection I think), but it is more convenience if this feature is integrated in driver.

[itamarhaber]

From what I know, max_idle_connections actually means how many connections should be kept open if idle. That option usually does not "preheat" the pool (e.g. https://github.com/gomodule/redigo/blob/master/redis/pool.go).

It looks like what @ethe is asking for is a min_connections option, which is a nice-to-have-feature and as per @andymccurdy's answer can be easily worked around. Granted, setting such an option to an absurdly high number, is misuse. I'd be happy to review a PR on the issue that adds this convenience :)

[theodesp]

Hey how can we implement this? Having a initial capacity property that will invoke the make_connection so many times and keep that number in the _available_connections ? If Yes I will be more than happy to submit a PR.

[antsbean]

Short answer, there's no automatic way to do this. You could do this manually by acquiring N number of connections, running a common on each of them, then releasing them all back to the connection pool.

why not do add max_idle_connections feature

[github-actions]

This issue is marked stale. It will be closed in 30 days if it is not updated.