Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Added celery pattern

  • Loading branch information...
commit 05f162329d425a3037c10cebfcdf941ce2802ca7 1 parent 3fdcefb
@mitsuhiko authored
Showing with 93 additions and 0 deletions.
  1. +92 −0 docs/patterns/celery.rst
  2. +1 −0  docs/patterns/index.rst
View
92 docs/patterns/celery.rst
@@ -0,0 +1,92 @@
+Celery Based Background Tasks
+=============================
+
+Celery is a task queue for Python with batteries included. It used to
+have a Flask integration but it became unnecessary after some
+restructuring of the internals of Celery with Version 3. This guide fills
+in the blanks in how to properly use Celery with Flask but assumes that
+you generally already read the `First Steps with Celery
+<http://docs.celeryproject.org/en/master/getting-started/first-steps-with-celery.html>`_
+guide in the official Celery documentation.
+
+Installing Celery
+-----------------
+
+Celery is on the Python Package Index (PyPI), so it can be installed with
+standard Python tools like ``pip`` or ``easy_install``::
+
+ $ pip install celery
+
+Configuring Celery
+------------------
+
+The first thing you need is a Celery instance, this is called the celery
+application. It serves the same purpose as the :class:`~flask.Flask`
+object in Flask, just for Celery. Since this instance is used as the
+entry-point for everything you want to do in Celery, like creating tasks
+and managing workers, it must be possible for other modules to import it.
+
+For instance you can place this in a ``tasks`` module. While you can use
+Celery without any reconfiguration with Flask, it becomes a bit nicer by
+subclassing tasks and adding support for Flask's application contexts and
+hooking it up with the Flask configuration.
+
+This is all that is necessary to properly integrate Celery with Flask::
+
+ from celery import Celery
+
+ def make_celery(app):
+ celery = Celery(app.import_name, broker=app.config['CELERY_BROKER_URL'])
+ celery.conf.update(app.config)
+ TaskBase = celery.Task
+ class ContextTask(TaskBase):
+ abstract = True
+ def __call__(self, *args, **kwargs):
+ with app.app_context():
+ return TaskBase.__call__(self, *args, **kwargs)
+ celery.Task = ContextTask
+ return celery
+
+The function creates a new Celery object, configures it with the broker
+from the application config, updates the rest of the Celery config from
+the Flask config and then creates a subclass of the task that wraps the
+task execution in an application context.
+
+Minimal Example
+---------------
+
+With what we have above this is the minimal example of using Celery with
+Flask::
+
+ from flask import Flask
+
+ app = Flask(__name__)
+ app.config.update(
+ CELERY_BROKER_URL='redis://localhost:6379',
+ CELERY_RESULT_BACKEND='redis://localhost:6379'
+ )
+ celery = make_celery(app)
+
+
+ @celery.task()
+ def add_together(a, b):
+ return a + b
+
+This task can now be called in the background:
+
+>>> result = add_together.delay(23, 42)
+>>> result.wait()
+65
+
+Running the Celery Worker
+-------------------------
+
+Now if you jumped in and already executed the above code you will be
+disappointed to learn that your ``.wait()`` will never actually return.
+That's because you also need to run celery. You can do that by running
+celery as a worker::
+
+ $ celery -A your_application worker
+
+The ``your_application`` string has to point to your application's package
+or module that creates the `celery` object.
View
1  docs/patterns/index.rst
@@ -39,3 +39,4 @@ Snippet Archives <http://flask.pocoo.org/snippets/>`_.
deferredcallbacks
methodoverrides
requestchecksum
+ celery
Please sign in to comment.
Something went wrong with that request. Please try again.