Changed default Twisted DNS Resolver to be blocking #642

wants to merge 1 commit into

2 participants


The default Twisted reactor uses a threaded DNS resolver by default (at least on platforms that support threading). As a result, if you're using a Twisted library, and you're looking up hostnames, all of the sudden your single threaded Tornado server will spawn additional threads.

This is kind of annoying because
1) Signals like Ctrl+C no longer work for the main Tornado process because the worker threads will be blocked on their request queues.
2) It's inconsistent with other Tornado functionality such as AsyncHttpClient which for better or worse, uses blocking dns lookups.
3) It's confusing - I dug through stuff for hours convinced I wasn't integrating Twisted + Tornado correctly.

So, I changed the default to make my life easier, but I can totally understand if you reject this, given that the Twisted behavior is both correct and more efficient. But, I figured I'd try to save others a few hours of time in case they suddenly get confused when Ctrl+c stops working in a Twisted+Tornado app.

tornadoweb member

Hmm. It's unfortunate that twisted's threadpool doesn't set the daemon flags on its threads. I don't want to discourage the use of threads like this. I'm introducing a threaded resolver for tornado 3.0, although I'm undecided about making it the default (my main concern is that you can't safely enter multi-process mode while background threads exist). These issues at least need to be documented better.

I haven't tried it, but I think if you call reactor.fireSystemEvent('shutdown') in a finally block after io_loop.start() (which will exit with a KeyboardInterrupt exception), Ctrl-C should work as expected. Or maybe it would work if you call instead of io_loop.start(). Unfortunately we don't have a good way to make this happen automatically since the IOLoop doesn't have a well-defined shutdown procedure (twisted's reactor can't be restarted once it's stopped, so they can shut down the thread pool once that's done, but it's possible to start and stop the tornado IOLoop repeatedly).

@bdarnell bdarnell closed this Apr 14, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment