Permalink
Browse files

Fill out docs about multiprocessing and eventlet

Closes #58
  • Loading branch information...
1 parent 43cbf0e commit 1697c31b445479067206d60f391f9ce9fbad24fc @thobbs thobbs committed Sep 14, 2011
Showing with 41 additions and 10 deletions.
  1. +1 −1 doc/index.rst
  2. +14 −7 doc/using_with/celery.rst
  3. +12 −0 doc/using_with/eventlet.rst
  4. +5 −2 doc/using_with/index.rst
  5. +9 −0 doc/using_with/multiprocessing.rst
View
2 doc/index.rst
@@ -23,7 +23,7 @@ Contents
How to work with various Cassandra and pycassa features.
:doc:`using_with/index`
- How to use pycassa with other projects, including Celery.
+ How to use pycassa with other projects, including eventlet and Celery.
:doc:`changelog`
The changelog for every version of pycassa.
View
21 doc/using_with/celery.rst
@@ -14,7 +14,7 @@ module is used to spawn new worker processes, while
`eventlet <http://eventlet.net/>`_ is used to spawn new worker green threads.
:mod:`multiprocessing`
-----------------------
+^^^^^^^^^^^^^^^^^^^^^^
The :class:`~.ConnectionPool` class is not :mod:`multiprocessing`-safe. Because
celery evaluates globals prior to spawning new worker processes, a global
:class:`~.ConnectionPool` will be shared among multiple processes. This is
@@ -26,14 +26,21 @@ you set
to 1 and start multiple separate celery processes. The process argument
``--pool=solo`` may also be used when starting the celery processes.
+.. seealso:: :ref:`using_with_multiprocessing`
+
:mod:`eventlet`
----------------
+^^^^^^^^^^^^^^^
Because the :class:`~.ConnectionPool` class uses concurrency primitives from
-the :mod:`threading` module, it should be possible to support
-:mod:`eventlet` worker threads after
+the :mod:`threading` module, you can use :mod:`eventlet` worker threads after
`monkey patching <http://eventlet.net/doc/basic_usage.html#patching-functions>`_
-the standard library. However, this option has not yet been well explored.
+the standard library. Specifically, the :mod:`threading` and :mod:`socket`
+modules must monkey-patched.
+
+Be aware that you may need to install `dnspython <http://pypi.python.org/pypi/dnspython>`_
+in order to connect to your nodes.
+
+.. seealso:: :ref:`using_with_eventlet`
Usage as a Broker Backend
-=========================
-**pycassa** is not currently a broker backend option.
+-------------------------
+pycassa is not currently a broker backend option.
View
12 doc/using_with/eventlet.rst
@@ -0,0 +1,12 @@
+.. _using_with_eventlet:
+
+Using with Eventlet
+===================
+Because the :class:`~.ConnectionPool` class uses concurrency primitives from
+the :mod:`threading` module, you can use :mod:`eventlet` green threads after
+`monkey patching <http://eventlet.net/doc/basic_usage.html#patching-functions>`_
+the standard library. Specifically, the :mod:`threading` and :mod:`socket`
+modules must monkey-patched.
+
+Be aware that you may need to install `dnspython <http://pypi.python.org/pypi/dnspython>`_
+in order to connect to your nodes.
View
7 doc/using_with/index.rst
@@ -1,6 +1,9 @@
-Using **pycassa** with Other Tools
-==================================
+Using pycassa with Other Tools
+==============================
.. toctree::
+ :maxdepth: 2
celery
+ eventlet
+ multiprocessing
View
9 doc/using_with/multiprocessing.rst
@@ -0,0 +1,9 @@
+.. _using_with_multiprocessing:
+
+Using with :mod:`multiprocessing`
+---------------------------------
+The :class:`~.ConnectionPool` class is not :mod:`multiprocessing`-safe.
+If you're using pycassa with multiprocessing, be sure to create one
+:class:`~.ConnectionPool` per process. Creating a :class:`~.ConnectionPool`
+before forking and sharing it among processes is inherently unsafe and will
+result in race conditions.

0 comments on commit 1697c31

Please sign in to comment.