added MONGO_MAX_POOL_SIZE and MONGO_USE_GREENLETS parameters #15

Closed
wants to merge 4 commits into
from

Projects

None yet

3 participants

@akhenakh
Contributor

I'm using flask-pymongo with gevent and this parameters are usefull for tuning.

@dcrosta dcrosta and 1 other commented on an outdated diff Nov 14, 2012
flask_pymongo/__init__.py
if auto_start_request not in (True, False):
raise TypeError('%s_AUTO_START_REQUEST must be a bool' % config_prefix)
+ if use_greenlets not in (True, False):
+ raise TypeError('%s_USE_GREENLETS must be a bool' % config_prefix)
+
+ max_pool_size
@dcrosta
dcrosta Nov 14, 2012 Owner

This line doesn't seem to be doing anything ... typo?

@akhenakh
akhenakh Nov 14, 2012 Contributor

oops fixed !

@dcrosta dcrosta and 1 other commented on an outdated diff Nov 14, 2012
flask_pymongo/__init__.py
@@ -215,6 +227,13 @@ def call_end_request(response):
return response
app.after_request(call_end_request)
+ if max_pool_size is not None:
+ try:
+ int(max_pool_size)
+ except ValueError:
+ raise TypeError('%s_MAX_POOL_SIZE must be an integer' % config_prefix)
+ kwargs['max_pool_size'] = max_pool_size
@dcrosta
dcrosta Nov 14, 2012 Owner

Please move this to where we do the other config validation, near line 170 or so.

@akhenakh
akhenakh Nov 14, 2012 Contributor

I'm not sure it is a good idea to be moved around 170 (the else clause of the uri connection) cause I have to do the same tests for the options, that's why I'm doing it after

@dcrosta
Owner
dcrosta commented Nov 14, 2012

Hey @ajdavis, I'm a little unfamiliar with how these arguments are intended to be used -- can you offer any insight as to whether this approach is workable and correct?

@ajdavis
ajdavis commented Nov 14, 2012

Hey Dan! Summary: max_pool_size is fine, but leave validation up to PyMongo. Don't expose use_greenlets.

max_pool_size is the horrifically-named option to control how many idle sockets the pool will keep open when they're returned to the pool (via end_request, or when a thread dies). So it's better thought of as "max idle sockets before I start closing any more that are returned to the pool."

It's far, far less frequently useful than anyone thinks. I'd venture to call it useless. People want it to mean "max concurrent connections allowed," but PyMongo has no such option. However, if you want to make max_pool_size configurable this is a fine way to do it.

I strongly recommend not validating it at all, since Connection or ReplicaSetConnection will much more thoroughly validate the parameter. Thus it's redundant and likely buggy to do your own validation here.

use_greenlets: please don't set this. We're removing it from public view very soon. Our idea was that you could either do gevent.monkey.patch_all(), or use gevent and set use_greenlets=True. We've changed our minds and we can't think of any good reason not to just do patch_all(), so we're going to legislate that and hide use_greenlets as best we can.

@akhenakh akhenakh removed USE_GREENLETS
let pymongo Connection validate MAX_POOL_SIZE
changed doc accordingly
238ed6e
@akhenakh
Contributor

Thanks @ajdavis and Dan

@dcrosta dcrosta commented on the diff Nov 15, 2012
docs/index.rst
@@ -83,6 +83,8 @@ directives:
:class:`~pymongo.connection.Connection`). When used
with PyMongo 2.1, this causes a warning to be
issued and has no effect. Default: ``True``.
+``MONGO_MAX_POOL_SIZE`` (optional): The maximum size limit for the connection pool.
+ Default: ``PyMongo default``.
@dcrosta
dcrosta Nov 15, 2012 Owner

Per @ajdavis we should probably note that this is the maximum number of idle connections that will be left open.

@dcrosta
Owner
dcrosta commented Nov 15, 2012

Looks great other than my one last nitpick comment.

@ajdavis
ajdavis commented Nov 15, 2012

Yeah. "The maximum size limit for the connection pool" is copied from PyMongo's docs, but that's a very unfortunately unclear description. I agree with @dcrosta .

@dcrosta dcrosta closed this in 98a34a3 Dec 15, 2012
@dcrosta
Owner
dcrosta commented Dec 15, 2012

Thanks! Sorry it took me a while to get around to merging this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment