Threading and concurrency

(source: http://groups.google.com/group/python-tornado/browse_thread/thread/bf4788a2fd45ad07)

There have been a lot of questions about threading on the mailing list lately so I just wanted to take this opportunity to remind everyone that multithreaded programming is notoriously difficult and should generally be a last resort. This is doubly true in python, where the GIL limits the benefits of multithreading (i.e. it doesn't help you take advantage of multiple cores).

If you use multiple threads, it is your responsibility to understand the thread-safety semantics of any libraries you use (this can be tricky as the relative rarity of threads in the python community means that thread-safety semantics are often undocumented). In the absence of specific evidence of thread-safety, assume that it's not safe to access shared objects directly or indirectly from multiple threads. In tornado, this often means the IOLoop. Nothing that refers to the IOLoop even indirectly can be used from other threads, except to use IOLoop.add_callback to transfer control back to the main thread.

In general, you should think about IO strategies for tornado apps in this order:

1. Do it synchronously and block the IOLoop. This is most appropriate for things like memcache and database queries that are under your control and should always be fast. If it's not fast, make it fast by adding the appropriate indexes to the database, etc.

2. Use an async library if available (e.g. AsyncHTTPClient). This comes below the synchronous option even though it would be kind of silly to use a synchronous HTTPClient in tornado, because the best outcome is for everything necessary to render a page to be so fast you don't mind blocking the IOLoop for it.

3. Move the work out of the tornado process. If you're sending email, for example, just write it to the database and let another process (whose latency doesn't matter) read from the queue and do the actual sending.

4. Do the work in a separate thread. Keep your threads' work units small - do a single synchronous operation and then hand the result back to the main thread (remember that you're not getting any CPU parallelism by doing more work on the other threads).

-Ben