Permalink
Browse files

Merge branch 'master' into kombu2

Conflicts:
	celery/app/amqp.py
	celery/messaging.py
	celery/task/base.py
	celery/utils/__init__.py
	celery/worker/__init__.py
	docs/configuration.rst
	docs/internals/worker.rst
	docs/reference/celery.conf.rst
	docs/tutorials/clickcounter.rst
  • Loading branch information...
2 parents c4d723d + 823f45c commit 0606fda57cac0c6c0538ee5d14c09056a273a5f4 Ask Solem committed Oct 28, 2010
Showing with 2,002 additions and 1,735 deletions.
  1. +373 −373 Changelog
  2. +30 −24 FAQ
  3. +7 −7 INSTALL
  4. +10 −10 README.rst
  5. +47 −54 celery/app/__init__.py
  6. +9 −1 celery/app/amqp.py
  7. +78 −113 celery/app/base.py
  8. +1 −0 celery/app/defaults.py
  9. +1 −1 celery/apps/beat.py
  10. +12 −2 celery/apps/worker.py
  11. +2 −2 celery/backends/base.py
  12. +1 −1 celery/backends/pyredis.py
  13. +1 −1 celery/backends/tyrant.py
  14. +1 −1 celery/beat.py
  15. +10 −10 celery/bin/camqadm.py
  16. +4 −4 celery/bin/celerybeat.py
  17. +22 −9 celery/bin/celeryd.py
  18. +8 −2 celery/concurrency/processes/__init__.py
  19. +42 −4 celery/concurrency/processes/pool.py
  20. +4 −4 celery/contrib/abortable.py
  21. +102 −34 celery/datastructures.py
  22. +4 −4 celery/db/a805d4bd.py
  23. +3 −3 celery/events/__init__.py
  24. +3 −3 celery/events/state.py
  25. +11 −1 celery/loaders/base.py
  26. +1 −1 celery/loaders/default.py
  27. +8 −8 celery/log.py
  28. +3 −3 celery/messaging.py
  29. +2 −2 celery/registry.py
  30. +50 −67 celery/result.py
  31. +4 −4 celery/routes.py
  32. +5 −5 celery/schedules.py
  33. +15 −17 celery/serialization.py
  34. +203 −242 celery/task/base.py
  35. +2 −2 celery/task/builtins.py
  36. +1 −1 celery/task/control.py
  37. +3 −3 celery/task/http.py
  38. +1 −1 celery/task/sets.py
  39. +3 −3 celery/tests/test_buckets.py
  40. +2 −1 celery/tests/test_worker_job.py
  41. +3 −3 celery/tests/utils.py
  42. +6 −7 celery/utils/__init__.py
  43. +4 −4 celery/utils/dispatch/saferef.py
  44. +6 −6 celery/utils/dispatch/signal.py
  45. +2 −2 celery/utils/functional.py
  46. +3 −3 celery/utils/timeutils.py
  47. +52 −56 celery/worker/__init__.py
  48. +33 −39 celery/worker/buckets.py
  49. +7 −4 celery/worker/consumer.py
  50. +11 −0 celery/worker/control/builtins.py
  51. +75 −15 celery/worker/controllers.py
  52. +6 −7 celery/worker/heartbeat.py
  53. +165 −151 celery/worker/job.py
  54. +20 −20 celery/worker/state.py
  55. +2 −2 contrib/debian/init.d/celerybeat
  56. +2 −2 contrib/debian/init.d/celeryd
  57. +2 −2 contrib/debian/init.d/celeryevcam
  58. +3 −3 contrib/generic-init.d/celeryd
  59. +4 −4 contrib/requirements/README.rst
  60. +45 −45 docs/configuration.rst
  61. +13 −13 docs/cookbook/daemonizing.rst
  62. +2 −2 docs/cookbook/tasks.rst
  63. +8 −8 docs/getting-started/broker-installation.rst
  64. +2 −2 docs/getting-started/first-steps-with-celery.rst
  65. +7 −7 docs/homepage/index.html
  66. +6 −6 docs/includes/installation.txt
  67. +9 −9 docs/includes/introduction.txt
  68. +3 −3 docs/includes/resources.txt
  69. +5 −5 docs/internals/app-overview.rst
  70. +8 −8 docs/internals/deprecation.rst
  71. +10 −10 docs/internals/protocol.rst
  72. +6 −6 docs/internals/worker.rst
  73. +1 −1 docs/links.rst
  74. +57 −0 docs/reference/celery.app.rst
  75. +37 −38 docs/reference/celery.conf.rst
  76. +2 −2 docs/reference/celery.signals.rst
  77. +1 −0 docs/reference/index.rst
  78. +11 −11 docs/releases/1.0/announcement.rst
  79. +21 −19 docs/tutorials/clickcounter.rst
  80. +2 −2 docs/tutorials/otherqueues.rst
  81. +27 −27 docs/userguide/executing.rst
  82. +36 −36 docs/userguide/monitoring.rst
  83. +16 −16 docs/userguide/periodic-tasks.rst
  84. +2 −2 docs/userguide/remote-tasks.rst
  85. +40 −40 docs/userguide/routing.rst
  86. +89 −25 docs/userguide/tasks.rst
  87. +6 −6 docs/userguide/tasksets.rst
  88. +15 −15 docs/userguide/workers.rst
  89. +4 −4 examples/celery_http_gateway/README.rst
  90. +6 −6 examples/ghetto-queue/README.rst
  91. +2 −2 examples/httpexample/README.rst
  92. +1 −1 examples/pythonproject/demoapp/README.rst
  93. +7 −0 setup.cfg
View
746 Changelog

Large diffs are not rendered by default.

Oops, something went wrong.
View
54 FAQ
@@ -130,8 +130,8 @@ Troubleshooting
MySQL is throwing deadlock errors, what can I do?
-------------------------------------------------
-**Answer:** MySQL has default isolation level set to ``REPEATABLE-READ``,
-if you don't really need that, set it to ``READ-COMMITTED``.
+**Answer:** MySQL has default isolation level set to `REPEATABLE-READ`,
+if you don't really need that, set it to `READ-COMMITTED`.
You can do that by adding the following to your :file:`my.cnf`::
[mysqld]
@@ -178,7 +178,7 @@ http://www.playingwithwire.com/2009/10/how-to-get-celeryd-to-work-on-freebsd/
.. _faq-duplicate-key-errors:
-I'm having ``IntegrityError: Duplicate Key`` errors. Why?
+I'm having `IntegrityError: Duplicate Key` errors. Why?
---------------------------------------------------------
**Answer:** See `MySQL is throwing deadlock errors, what can I do?`_.
@@ -284,7 +284,7 @@ Results
How do I get the result of a task if I have the ID that points there?
----------------------------------------------------------------------
-**Answer**: Use ``Task.AsyncResult``::
+**Answer**: Use `Task.AsyncResult`::
>>> result = MyTask.AsyncResult(task_id)
>>> result.get()
@@ -340,7 +340,7 @@ as a message. If you don't collect these results, they will build up and
RabbitMQ will eventually run out of memory.
If you don't use the results for a task, make sure you set the
-``ignore_result`` option:
+`ignore_result` option:
.. code-block python
@@ -383,14 +383,14 @@ The STOMP carrot backend requires the `stompy`_ library::
.. _`stompy`: http://pypi.python.org/pypi/stompy
-In this example we will use a queue called ``celery`` which we created in
+In this example we will use a queue called `celery` which we created in
the ActiveMQ web admin interface.
-**Note**: When using ActiveMQ the queue name needs to have ``"/queue/"``
-prepended to it. i.e. the queue ``celery`` becomes ``/queue/celery``.
+**Note**: When using ActiveMQ the queue name needs to have `"/queue/"`
+prepended to it. i.e. the queue `celery` becomes `/queue/celery`.
Since STOMP doesn't have exchanges and the routing capabilities of AMQP,
-you need to set ``exchange`` name to the same as the queue name. This is
+you need to set `exchange` name to the same as the queue name. This is
a minor inconvenience since carrot needs to maintain the same interface
for both AMQP and STOMP.
@@ -474,11 +474,17 @@ For more information see :ref:`task-request-info`.
Can I specify a custom task_id?
-------------------------------
-**Answer**: Yes. Use the ``task_id`` argument to
+**Answer**: Yes. Use the `task_id` argument to
:meth:`~celery.execute.apply_async`::
>>> task.apply_async(args, kwargs, task_id="...")
+
+Can I use decorators with tasks?
+--------------------------------
+
+**Answer**: Yes. But please see note at :ref:`tasks-decorating`.
+
.. _faq-natural-task-ids:
Can I use natural task ids?
@@ -523,7 +529,7 @@ See :doc:`userguide/tasksets` for more information.
Can I cancel the execution of a task?
-------------------------------------
-**Answer**: Yes. Use ``result.revoke``::
+**Answer**: Yes. Use `result.revoke`::
>>> result = add.apply_async(args=[2, 2], countdown=120)
>>> result.revoke()
@@ -566,8 +572,8 @@ See :doc:`userguide/routing` for more information.
Can I change the interval of a periodic task at runtime?
--------------------------------------------------------
-**Answer**: Yes. You can override ``PeriodicTask.is_due`` or turn
-``PeriodicTask.run_every`` into a property:
+**Answer**: Yes. You can override `PeriodicTask.is_due` or turn
+`PeriodicTask.run_every` into a property:
.. code-block:: python
@@ -601,11 +607,11 @@ Should I use retry or acks_late?
**Answer**: Depends. It's not necessarily one or the other, you may want
to use both.
-``Task.retry`` is used to retry tasks, notably for expected errors that
-is catchable with the ``try:`` block. The AMQP transaction is not used
+`Task.retry` is used to retry tasks, notably for expected errors that
+is catchable with the `try:` block. The AMQP transaction is not used
for these errors: **if the task raises an exception it is still acked!**.
-The ``acks_late`` setting would be used when you need the task to be
+The `acks_late` setting would be used when you need the task to be
executed again if the worker (for some reason) crashes mid-execution.
It's important to note that the worker is not known to crash, and if
it does it is usually an unrecoverable error that requires human
@@ -631,11 +637,11 @@ It's a good default, users who require it and know what they
are doing can still enable acks_late (and in the future hopefully
use manual acknowledgement)
-In addition ``Task.retry`` has features not available in AMQP
+In addition `Task.retry` has features not available in AMQP
transactions: delay between retries, max retries, etc.
So use retry for Python errors, and if your task is reentrant
-combine that with ``acks_late`` if that level of reliability
+combine that with `acks_late` if that level of reliability
is required.
.. _faq-schedule-at-specific-time:
@@ -645,7 +651,7 @@ Can I schedule tasks to execute at a specific time?
.. module:: celery.task.base
-**Answer**: Yes. You can use the ``eta`` argument of :meth:`Task.apply_async`.
+**Answer**: Yes. You can use the `eta` argument of :meth:`Task.apply_async`.
Or to schedule a periodic task at a specific time, use the
:class:`celery.task.schedules.crontab` schedule behavior:
@@ -662,7 +668,7 @@ Or to schedule a periodic task at a specific time, use the
.. _faq-safe-worker-shutdown:
-How do I shut down ``celeryd`` safely?
+How do I shut down `celeryd` safely?
--------------------------------------
**Answer**: Use the :sig:`TERM` signal, and the worker will finish all currently
@@ -672,7 +678,7 @@ You should never stop :mod:`~celery.bin.celeryd` with the :sig:`KILL` signal
(:option:`-9`), unless you've tried :sig:`TERM` a few times and waited a few
minutes to let it get a chance to shut down. As if you do tasks may be
terminated mid-execution, and they will not be re-run unless you have the
-``acks_late`` option set (``Task.acks_late`` / :setting:`CELERY_ACKS_LATE`).
+`acks_late` option set (`Task.acks_late` / :setting:`CELERY_ACKS_LATE`).
.. seealso::
@@ -705,14 +711,14 @@ See http://bit.ly/bo9RSw
.. _faq-windows-worker-embedded-beat:
-The ``-B`` / ``--beat`` option to celeryd doesn't work?
+The `-B` / `--beat` option to celeryd doesn't work?
----------------------------------------------------------------
-**Answer**: That's right. Run ``celerybeat`` and ``celeryd`` as separate
+**Answer**: That's right. Run `celerybeat` and `celeryd` as separate
services instead.
.. _faq-windows-django-settings:
-``django-celery`` can’t find settings?
+`django-celery` can’t find settings?
--------------------------------------
**Answer**: You need to specify the :option:`--settings` argument to
View
14 INSTALL
@@ -1,19 +1,19 @@
-Installing celery
+Installing Celery
=================
-You can install ``celery`` either via the Python Package Index (PyPI)
+You can install Celery either via the Python Package Index (PyPI)
or from source.
-To install using ``pip``,::
+To install using `pip`::
- $ pip install celery
+ $ pip install Celery
-To install using ``easy_install``,::
+To install using `easy_install`::
- $ easy_install celery
+ $ easy_install Celery
If you have downloaded a source tarball you can install it
-by doing the following,::
+by doing the following::
$ python setup.py build
# python setup.py install # as root
View
@@ -41,7 +41,7 @@ the `django-celery`_, `celery-pylons`_ and `Flask-Celery`_ add-on packages.
.. _`Pylons`: http://pylonshq.com/
.. _`Flask`: http://flask.pocoo.org/
.. _`django-celery`: http://pypi.python.org/pypi/django-celery
-.. _`celery-pylons`: http://bitbucket.org/ianschenck/celery-pylons
+.. _`celery-pylons`: http://pypi.python.org/pypi/celery-pylons
.. _`Flask-Celery`: http://github.com/ask/flask-celery/
.. _`operate with other languages using webhooks`:
http://ask.github.com/celery/userguide/remote-tasks.html
@@ -59,7 +59,7 @@ This is a high level overview of the architecture.
.. image:: http://cloud.github.com/downloads/ask/celery/Celery-Overview-v4.jpg
The broker delivers tasks to the worker servers.
-A worker server is a networked machine running ``celeryd``. This can be one or
+A worker server is a networked machine running `celeryd`. This can be one or
more machines depending on the workload.
The result of the task can be stored for later retrieval (called its
@@ -107,7 +107,7 @@ Features
| | while the queue is temporarily overloaded). |
+-----------------+----------------------------------------------------+
| Concurrency | Tasks are executed in parallel using the |
- | | ``multiprocessing`` module. |
+ | | `multiprocessing` module. |
+-----------------+----------------------------------------------------+
| Scheduling | Supports recurring tasks like cron, or specifying |
| | an exact date or countdown for when after the task |
@@ -194,14 +194,14 @@ is hosted at Github.
Installation
============
-You can install ``celery`` either via the Python Package Index (PyPI)
+You can install `celery` either via the Python Package Index (PyPI)
or from source.
-To install using ``pip``,::
+To install using `pip`,::
$ pip install celery
-To install using ``easy_install``,::
+To install using `easy_install`,::
$ easy_install celery
@@ -210,7 +210,7 @@ To install using ``easy_install``,::
Downloading and installing from source
--------------------------------------
-Download the latest version of ``celery`` from
+Download the latest version of `celery` from
http://pypi.python.org/pypi/celery/
You can install it by doing the following,::
@@ -275,18 +275,18 @@ http://wiki.github.com/ask/celery/
Contributing
============
-Development of ``celery`` happens at Github: http://github.com/ask/celery
+Development of `celery` happens at Github: http://github.com/ask/celery
You are highly encouraged to participate in the development
-of ``celery``. If you don't like Github (for some reason) you're welcome
+of `celery`. If you don't like Github (for some reason) you're welcome
to send regular patches.
.. _license:
License
=======
-This software is licensed under the ``New BSD License``. See the ``LICENSE``
+This software is licensed under the `New BSD License`. See the ``LICENSE``
file in the top distribution directory for the full license text.
.. # vim: syntax=rst expandtab tabstop=4 shiftwidth=4 shiftround
Oops, something went wrong.

0 comments on commit 0606fda

Please sign in to comment.