Permalink
Browse files

Fix a memory leak in stack_context.

The old_contexts reference in StackContexts could maintain a chain of
old irrelevant contexts, so clear it once it's no longer needed.
This was mainly a problem in gen.engine, where additional contexts
would accumulate in memory (but not on the stack) for each asynchronous
operation.

Also clear the deactivate_stack_context in gen.Runner to allow
the StackContext to be garbage-collected sooner.
  • Loading branch information...
bdarnell committed Nov 17, 2012
1 parent b724652 commit bff07405549a6eb173a4cfc9bbc3fc7c6da5cdd7
Showing with 24 additions and 0 deletions.
  1. +1 −0 tornado/gen.py
  2. +1 −0 tornado/stack_context.py
  3. +22 −0 tornado/test/gen_test.py
View
@@ -374,6 +374,7 @@ def run(self):
"finished without waiting for callbacks %r" %
self.pending_callbacks)
self.deactivate_stack_context()
+ self.deactivate_stack_context = None
return
except Exception:
self.finished = True
View
@@ -160,6 +160,7 @@ def __exit__(self, type, value, traceback):
return self.exception_handler(type, value, traceback)
finally:
_state.contexts = self.old_contexts
+ self.old_contexts = None
class NullContext(object):
View
@@ -270,6 +270,28 @@ def outer():
initial_stack_depth = len(stack_context._state.contexts)
self.run_gen(outer)
+ def test_stack_context_leak_exception(self):
+ # same as previous, but with a function that exits with an exception
+ from tornado import stack_context
+
+ @gen.engine
+ def inner(callback):
+ yield gen.Task(self.io_loop.add_callback)
+ 1 / 0
+
+ @gen.engine
+ def outer():
+ for i in xrange(10):
+ try:
+ yield gen.Task(inner)
+ except ZeroDivisionError:
+ pass
+ stack_increase = len(stack_context._state.contexts) - initial_stack_depth
+ self.assertTrue(stack_increase <= 2)
+ self.stop()
+ initial_stack_depth = len(stack_context._state.contexts)
+ self.run_gen(outer)
+
class GenSequenceHandler(RequestHandler):
@asynchronous

4 comments on commit bff0740

@mchruszcz

This comment has been minimized.

Show comment Hide comment
@mchruszcz

mchruszcz Nov 20, 2012

This is a very important fix and I would love to see it in the next stable version as soon as possible. Is there an approximate date I could expect this to be released?

This is a very important fix and I would love to see it in the next stable version as soon as possible. Is there an approximate date I could expect this to be released?

@bdarnell

This comment has been minimized.

Show comment Hide comment
@bdarnell

bdarnell Nov 22, 2012

Owner

I think the next full release (3.0) is still a ways off, so I'll try to put together a 2.4.1 bugfix release in the next week or so with this and other bug fixes.

Owner

bdarnell replied Nov 22, 2012

I think the next full release (3.0) is still a ways off, so I'll try to put together a 2.4.1 bugfix release in the next week or so with this and other bug fixes.

@bdarnell

This comment has been minimized.

Show comment Hide comment
@bdarnell

bdarnell Nov 25, 2012

Owner

I've just released 2.4.1.

Owner

bdarnell replied Nov 25, 2012

I've just released 2.4.1.

@mchruszcz

This comment has been minimized.

Show comment Hide comment
@mchruszcz

mchruszcz Nov 25, 2012

That's good news! Thanks a lot.

That's good news! Thanks a lot.

Please sign in to comment.