Skip to content

Commit

Permalink
Added celery pattern
Browse files Browse the repository at this point in the history
  • Loading branch information
mitsuhiko committed Jan 27, 2013
1 parent 3fdcefb commit 05f1623
Show file tree
Hide file tree
Showing 2 changed files with 93 additions and 0 deletions.
92 changes: 92 additions & 0 deletions docs/patterns/celery.rst
@@ -0,0 +1,92 @@
Celery Based Background Tasks
=============================

Celery is a task queue for Python with batteries included. It used to
have a Flask integration but it became unnecessary after some
restructuring of the internals of Celery with Version 3. This guide fills
in the blanks in how to properly use Celery with Flask but assumes that
you generally already read the `First Steps with Celery
<http://docs.celeryproject.org/en/master/getting-started/first-steps-with-celery.html>`_
guide in the official Celery documentation.

Installing Celery
-----------------

Celery is on the Python Package Index (PyPI), so it can be installed with
standard Python tools like ``pip`` or ``easy_install``::

$ pip install celery

Configuring Celery
------------------

The first thing you need is a Celery instance, this is called the celery
application. It serves the same purpose as the :class:`~flask.Flask`
object in Flask, just for Celery. Since this instance is used as the
entry-point for everything you want to do in Celery, like creating tasks
and managing workers, it must be possible for other modules to import it.

For instance you can place this in a ``tasks`` module. While you can use
Celery without any reconfiguration with Flask, it becomes a bit nicer by
subclassing tasks and adding support for Flask's application contexts and
hooking it up with the Flask configuration.

This is all that is necessary to properly integrate Celery with Flask::

from celery import Celery

def make_celery(app):
celery = Celery(app.import_name, broker=app.config['CELERY_BROKER_URL'])
celery.conf.update(app.config)
TaskBase = celery.Task
class ContextTask(TaskBase):
abstract = True
def __call__(self, *args, **kwargs):
with app.app_context():
return TaskBase.__call__(self, *args, **kwargs)
celery.Task = ContextTask
return celery

The function creates a new Celery object, configures it with the broker
from the application config, updates the rest of the Celery config from
the Flask config and then creates a subclass of the task that wraps the
task execution in an application context.

Minimal Example
---------------

With what we have above this is the minimal example of using Celery with
Flask::

from flask import Flask

app = Flask(__name__)
app.config.update(
CELERY_BROKER_URL='redis://localhost:6379',
CELERY_RESULT_BACKEND='redis://localhost:6379'
)
celery = make_celery(app)


@celery.task()
def add_together(a, b):
return a + b

This task can now be called in the background:

>>> result = add_together.delay(23, 42)
>>> result.wait()
65

Running the Celery Worker
-------------------------

Now if you jumped in and already executed the above code you will be
disappointed to learn that your ``.wait()`` will never actually return.
That's because you also need to run celery. You can do that by running
celery as a worker::

$ celery -A your_application worker

The ``your_application`` string has to point to your application's package
or module that creates the `celery` object.
1 change: 1 addition & 0 deletions docs/patterns/index.rst
Expand Up @@ -39,3 +39,4 @@ Snippet Archives <http://flask.pocoo.org/snippets/>`_.
deferredcallbacks
methodoverrides
requestchecksum
celery

0 comments on commit 05f1623

Please sign in to comment.