Skip to content


Subversion checkout URL

You can clone with
Download ZIP


Make gen_test's timeout configurable #723

merged 5 commits into from

3 participants


Lessons learned from testing Motor with my equivalent, async_test_engine: it's nice to be able to increase the timeout on specific tests if they legitimately take over 5 seconds. It's also nice to set the timeout with an environment variable when launching an interactive debugging session - in this case I generally set the timeout to something huge so I'm not interrupted while debugging.


This generally seems like a good idea, but I have a few comments:

For decorators with optional arguments, I prefer to have "func=None" as the first argument and require that the other args be passed by keyword, both to enforce use of the keyword at the call site for legibility and to avoid type-based dispatch.

I'm -1 on using an environment variable to set the timeout process-wide (especially with as generic a name as "TIMEOUT"). If we're going to have a global config I'd rather figure out how best to use tornado.options for this kind of internal use, but setting a global floor in absolute terms doesn't feel like the right approach.

If your use case is a debugger I think what you want is not a floor on all timeouts but a way to suspend time while the program is paused. Fortunately I think that's not too hard as long as your code uses IOLoop.time() instead of time.time() (BTW, the toro 0.5 docs talk about time.time() but the code uses IOLoop.time()).

class DebuggableIOLoop(SelectIOLoop):
  def __init__(self):
    self.clock = 0

  def poll(self, timeout):
    start = time.time()
    result = super().poll(timeout)
    self.clock += time.time() - start
    return result

  def time(self):
    return self.clock

I'll fix the Toro docs, thanks. And I agree about the decorator's arguments.

I think I understand your DebuggableIOLoop idea, interesting.

I assert that interactively debugging a gen_test-decorated method is a very common use-case and that overriding the timeout should be an easily accessible feature. When a test fails I want to start debugging it immediately. IDEs like PyCharm make it easy to set an environment variable before debugging. Furthermore, PyCharm knows how to run the test my cursor is in right now, a feature I use all the time—but it runs it with Nose, not tornado.test.runtests. Hence my preference for the env var.

Which do you think is best?:
1. Keep the env var, but give it a better name (GEN_TEST_TIMEOUT?)
2. Make AsyncTestCase use DebuggableIOLoop by default
3. Ship DebuggableIOLoop so it's available with runtests --ioloop=DebuggableIOLoop (makes it harder to use with IDEs)
4. Something else?


Interesting. Many of the python users I know left java to get away from the IDE-centric culture there, so I haven't heard much about what python IDEs do. (dictating a particular test runner seems kind of obnoxious)

Whatever we do for gen_test's timeout should also apply to AsyncTestCase.wait, so if it's an environment variable maybe call it ASYNC_TEST_TIMEOUT?

DebuggableIOLoop is a little too magical to make the default (plus it's not really usable by projects that haven't dropped Tornado 2.x compatibility, since it requires the use of IOLoop.time).

I think the viable options are:
1. Environment variable. It's a small change and most compatible with the IDE, but it introduces a different kind of configuration than we use anywhere else.
2. Overridable method on AsyncTestCase. This allows application developers to override timeouts however they want (environment variable? detect debugger with sys.gettrace?) and gets it out of my hair, but projects have to opt in if any of their developers want to use this feature. It's consistent with methods like get_new_ioloop, but less convenient.
3. tornado.options. Overridable constants like this are what the options module is good at but we don't currently impose the options module on projects, and it's unclear how you'd set the option in this case.

I'll think some more about the future of the options module and whether it makes sense to use it here. Otherwise I'm leaning towards the environment variable.


Thanks, Ben. I agree that PyCharm's test config is inflexible, but even so it's awfully convenient! I'll show you some time. A quick Googling suggests that Komodo's and WingWare's testing features are similarly useful and restrictive.

The latest patches rename the variable to ASYNC_TEST_TIMEOUT and apply it to AsyncTestCase.wait as well as gen_test.


Been following along. I'm excited about this change because I also use PyCharm and have run into some of the same frustration as Jesse. I wanted to note that you have your choice of what test runner to use with the IDE. I too prefer nosetest, but I believe other test runners are usable as well.


OK, I've decided to merge this, and then to address my concerns about environment vs tornado.options I'm going to make the timeout an option that takes its default variable from an environment variable. The next question is whether all options should look for a similarly-named environment variable. Have you seen a need for other settings to be configured via the environment?

I think it might be useful to have e.g. a --ioloop option with an environment variable default; this would help in cases like #615 where a user needs to select a non-default IOLoop for all tornado processes they run (other hypothetical options that would be useful in both a command-line and environment form are --httpclient_allow_ipv6 and the proxy-related options).

One drawback to reading from environment variables is that options that rely on hooks (e.g. logging) would still require parse_command_line or parse_config_file to be run at some point to trigger the hooks so an env-only configuration wouldn't work consistently.

@bdarnell bdarnell merged commit 0d44009 into tornadoweb:master

1 check passed

Details default The Travis build passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
Showing with 133 additions and 10 deletions.
  1. +82 −1 tornado/test/
  2. +51 −9 tornado/
83 tornado/test/
@@ -2,10 +2,27 @@
from __future__ import absolute_import, division, print_function, with_statement
-from tornado import gen
+from tornado import gen, ioloop
from tornado.testing import AsyncTestCase, gen_test
from tornado.test.util import unittest
+import contextlib
+import os
+def set_environ(name, value):
+ old_value = os.environ.get('name')
+ os.environ[name] = value
+ try:
+ yield
+ finally:
+ if old_value is None:
+ del os.environ[name]
+ else:
+ os.environ[name] = old_value
class AsyncTestCaseTest(AsyncTestCase):
def test_exception_in_callback(self):
@@ -16,6 +33,24 @@ def test_exception_in_callback(self):
except ZeroDivisionError:
+ def test_wait_timeout(self):
+ time = self.io_loop.time
+ # Accept default 5-second timeout, no error
+ self.io_loop.add_timeout(time() + 0.01, self.stop)
+ self.wait()
+ # Timeout passed to wait()
+ self.io_loop.add_timeout(time() + 1, self.stop)
+ with self.assertRaises(self.failureException):
+ self.wait(timeout=0.01)
+ # Timeout set with environment variable
+ self.io_loop.add_timeout(time() + 1, self.stop)
+ with set_environ('ASYNC_TEST_TIMEOUT', '0.01'):
+ with self.assertRaises(self.failureException):
+ self.wait()
def test_subsequent_wait_calls(self):
This test makes sure that a second call to wait()
@@ -74,5 +109,51 @@ def test_async(self):
yield gen.Task(self.io_loop.add_callback)
self.finished = True
+ def test_timeout(self):
+ # Set a short timeout and exceed it.
+ @gen_test(timeout=0.1)
+ def test(self):
+ yield gen.Task(self.io_loop.add_timeout, self.io_loop.time() + 1)
+ with self.assertRaises(ioloop.TimeoutError):
+ test(self)
+ self.finished = True
+ def test_no_timeout(self):
+ # A test that does not exceed its timeout should succeed.
+ @gen_test(timeout=1)
+ def test(self):
+ time = self.io_loop.time
+ yield gen.Task(self.io_loop.add_timeout, time() + 0.1)
+ test(self)
+ self.finished = True
+ def test_timeout_environment_variable(self):
+ @gen_test(timeout=0.5)
+ def test_long_timeout(self):
+ time = self.io_loop.time
+ yield gen.Task(self.io_loop.add_timeout, time() + 0.25)
+ # Uses provided timeout of 0.5 seconds, doesn't time out.
+ with set_environ('ASYNC_TEST_TIMEOUT', '0.1'):
+ test_long_timeout(self)
+ self.finished = True
+ def test_no_timeout_environment_variable(self):
+ @gen_test(timeout=0.01)
+ def test_short_timeout(self):
+ time = self.io_loop.time
+ yield gen.Task(self.io_loop.add_timeout, time() + 1)
+ # Uses environment-variable timeout of 0.1, times out.
+ with set_environ('ASYNC_TEST_TIMEOUT', '0.1'):
+ with self.assertRaises(ioloop.TimeoutError):
+ test_short_timeout(self)
+ self.finished = True
if __name__ == '__main__':
60 tornado/
@@ -82,6 +82,17 @@ def bind_unused_port():
return sock, port
+def get_async_test_timeout():
+ """Get the global timeout setting for async tests.
+ Returns a float, the timeout in seconds.
+ """
+ try:
+ return float(os.environ.get('ASYNC_TEST_TIMEOUT'))
+ except (ValueError, TypeError):
+ return 5
class AsyncTestCase(unittest.TestCase):
"""`~unittest.TestCase` subclass for testing `.IOLoop`-based
asynchronous code.
@@ -202,14 +213,19 @@ def stop(self, _arg=None, **kwargs):
self.__running = False
self.__stopped = True
- def wait(self, condition=None, timeout=5):
+ def wait(self, condition=None, timeout=None):
"""Runs the `.IOLoop` until stop is called or timeout has passed.
- In the event of a timeout, an exception will be thrown.
+ In the event of a timeout, an exception will be thrown. The default
+ timeout is 5 seconds; it may be overridden with a ``timeout`` keyword
+ argument or globally with the ASYNC_TEST_TIMEOUT environment variable.
If ``condition`` is not None, the `.IOLoop` will be restarted
after `stop()` until ``condition()`` returns true.
+ if timeout is None:
+ timeout = get_async_test_timeout()
if not self.__stopped:
if timeout:
def timeout_func():
@@ -354,7 +370,7 @@ def get_protocol(self):
return 'https'
-def gen_test(f):
+def gen_test(func=None, timeout=None):
"""Testing equivalent of ``@gen.coroutine``, to be applied to test methods.
``@gen.coroutine`` cannot be used on tests because the `.IOLoop` is not
@@ -368,13 +384,39 @@ class MyTest(AsyncHTTPTestCase):
def test_something(self):
response = yield gen.Task(self.fetch('/'))
- """
- f = gen.coroutine(f)
+ By default, ``@gen_test`` times out after 5 seconds. The timeout may be
+ overridden globally with the ASYNC_TEST_TIMEOUT environment variable,
+ or for each test with the ``timeout`` keyword argument:
- @functools.wraps(f)
- def wrapper(self):
- return self.io_loop.run_sync(functools.partial(f, self), timeout=5)
- return wrapper
+ class MyTest(AsyncHTTPTestCase):
+ @gen_test(timeout=10)
+ def test_something_slow(self):
+ response = yield gen.Task(self.fetch('/'))
+ If both the environment variable and the parameter are set, ``gen_test``
+ uses the maximum of the two.
+ """
+ if timeout is None:
+ timeout = get_async_test_timeout()
+ def wrap(f):
+ f = gen.coroutine(f)
+ @functools.wraps(f)
+ def wrapper(self):
+ return self.io_loop.run_sync(
+ functools.partial(f, self), timeout=timeout)
+ return wrapper
+ if func is not None:
+ # Used like:
+ # @gen_test
+ # def f(self):
+ # pass
+ return wrap(func)
+ else:
+ # Used like @gen_test(timeout=10)
+ return wrap
# Without this attribute, nosetests will try to run gen_test as a test
Something went wrong with that request. Please try again.