Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
2 changed files
with
93 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
Celery Based Background Tasks | ||
============================= | ||
|
||
Celery is a task queue for Python with batteries included. It used to | ||
have a Flask integration but it became unnecessary after some | ||
restructuring of the internals of Celery with Version 3. This guide fills | ||
in the blanks in how to properly use Celery with Flask but assumes that | ||
you generally already read the `First Steps with Celery | ||
<http://docs.celeryproject.org/en/master/getting-started/first-steps-with-celery.html>`_ | ||
guide in the official Celery documentation. | ||
|
||
Installing Celery | ||
----------------- | ||
|
||
Celery is on the Python Package Index (PyPI), so it can be installed with | ||
standard Python tools like ``pip`` or ``easy_install``:: | ||
|
||
$ pip install celery | ||
|
||
Configuring Celery | ||
------------------ | ||
|
||
The first thing you need is a Celery instance, this is called the celery | ||
application. It serves the same purpose as the :class:`~flask.Flask` | ||
object in Flask, just for Celery. Since this instance is used as the | ||
entry-point for everything you want to do in Celery, like creating tasks | ||
and managing workers, it must be possible for other modules to import it. | ||
|
||
For instance you can place this in a ``tasks`` module. While you can use | ||
Celery without any reconfiguration with Flask, it becomes a bit nicer by | ||
subclassing tasks and adding support for Flask's application contexts and | ||
hooking it up with the Flask configuration. | ||
|
||
This is all that is necessary to properly integrate Celery with Flask:: | ||
|
||
from celery import Celery | ||
|
||
def make_celery(app): | ||
celery = Celery(app.import_name, broker=app.config['CELERY_BROKER_URL']) | ||
celery.conf.update(app.config) | ||
TaskBase = celery.Task | ||
class ContextTask(TaskBase): | ||
abstract = True | ||
def __call__(self, *args, **kwargs): | ||
with app.app_context(): | ||
return TaskBase.__call__(self, *args, **kwargs) | ||
celery.Task = ContextTask | ||
return celery | ||
|
||
The function creates a new Celery object, configures it with the broker | ||
from the application config, updates the rest of the Celery config from | ||
the Flask config and then creates a subclass of the task that wraps the | ||
task execution in an application context. | ||
|
||
Minimal Example | ||
--------------- | ||
|
||
With what we have above this is the minimal example of using Celery with | ||
Flask:: | ||
|
||
from flask import Flask | ||
|
||
app = Flask(__name__) | ||
app.config.update( | ||
CELERY_BROKER_URL='redis://localhost:6379', | ||
CELERY_RESULT_BACKEND='redis://localhost:6379' | ||
) | ||
celery = make_celery(app) | ||
|
||
|
||
@celery.task() | ||
def add_together(a, b): | ||
return a + b | ||
|
||
This task can now be called in the background: | ||
|
||
>>> result = add_together.delay(23, 42) | ||
>>> result.wait() | ||
65 | ||
|
||
Running the Celery Worker | ||
------------------------- | ||
|
||
Now if you jumped in and already executed the above code you will be | ||
disappointed to learn that your ``.wait()`` will never actually return. | ||
That's because you also need to run celery. You can do that by running | ||
celery as a worker:: | ||
|
||
$ celery -A your_application worker | ||
|
||
The ``your_application`` string has to point to your application's package | ||
or module that creates the `celery` object. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -39,3 +39,4 @@ Snippet Archives <http://flask.pocoo.org/snippets/>`_. | |
deferredcallbacks | ||
methodoverrides | ||
requestchecksum | ||
celery |