Documentation Request: Redis Connection Pool #2

Closed
whardier opened this Issue Feb 23, 2012 · 11 comments

Projects

None yet

3 participants

The following works and is worth noting. This is IMPORTANT to keep pycket from reconnecting for EVERY request which can eat up your ulimit very quickly. I tested this by making a sample app, threading out 10 times, then using ab with 100000 requests and 100 concurrent requests.

Also.. this makes good use of the pass through settings that pycket does to the driver.

pycket_pool = redis.ConnectionPool(max_connections=10, host=options.redis_host, port=options.redis_port)
#...
            **{
                'pycket': {
                    'engine': 'redis',
                    'storage': {
                        'connection_pool': pycket_pool,
                        'db_sessions': 10,
                        'db_notifications': 11,
                    },
                    'cookies': {
                        'expires_days': 120,
                    },
                },
            }
#...
Owner

Nice, whardier, thanks a lot! I'll document this ASAP! :-)

Cheers,

Diogo

It seems that since tornado doesn't thread.. setting the pool
max_connection count to 1 works just fine :) It at least keeps the
connection open.. you may have to deal with problems where the
connection no longer exists.. so it passes a lot of the control and
responsibility back to the Application request handler.

On Thu, Feb 23, 2012 at 1:07 PM, Diogo Baeder
reply@reply.github.com
wrote:

Nice, whardier, thanks a lot! I'll document this ASAP! :-)

Cheers,

Diogo


Reply to this email directly or view it on GitHub:
#2 (comment)

Owner

Yep, but at least they're alternatives to avoid connection leak.

But besides all this flexibility, I'm getting a little uncomfortable of exposing so much of the storage backend to the client code that uses pycket. This just increases coupling. :-(

Nevertheless, it's important to document.

Understood

On Thu, Feb 23, 2012 at 4:02 PM, Diogo Baeder
reply@reply.github.com
wrote:

Yep, but at least they're alternatives to avoid connection leak.

But besides all this flexibility, I'm getting a little uncomfortable of exposing so much of the storage backend to the client code that uses pycket. This just increases coupling. :-(

Nevertheless, it's important to document.


Reply to this email directly or view it on GitHub:
#2 (comment)

just to note
it seems db numbers (db_sessions and db_notifications) are useless in config in case ConnectionPool is in action

according to https://github.com/andymccurdy/redis-py/blob/master/redis/client.py#L266 we must set db number while initializing ConnectionPool

it's easy to check this using interactive shell:

>>> import redis
>>> r = redis.Redis(db=10)  # configured to store db_sessions in settings using ConnectionPool
>>> r.keys()  # will show empty list (but there is at least one session)
[]
>>> r = redis.Redis(db=0)  # not configured to store db_sessions
>>> r.keys()
['305a7fe4-1999-4593-8a6e-2cfa90226d31', '7faf3f99-b339-4407-98e8-4a425344cce8']
>>>

thank you for nice library! :)

Owner

Ah, nice, thanks, man! I opened this issue, to implement it:
#5

I'll see when I can do it, but feel free to contribute with this via pull request. (Remember to test everything!)

Owner

Fixed:
e79fc19

Will be available soon in the next release (today or tomorrow). Bonus: pycket will now support Python 3. ;-)

Great, thank you, Diogo! :)

Owner

No worries, pal! Thanks for having reminded me of this problem. :-)

Also, make sure to update pycket, I've released the new version today, with this fix.

Also, make sure to update pycket, I've released the new version today, with this fix.

done)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment