methods as tasks fail with CELERY_ALWAYS_EAGER=True

[Karmak23]

Hi,

I've got this stacktrace when I run tests of my webapp:

Traceback (most recent call last):
  File "/Users/olive/sources/1flow/oneflow/core/tests/test_models.py", line 219, in setUp
    url='http://test.com/test1').save()
  File "/Users/olive/.virtualenvs/1flow/lib/python2.7/site-packages/mongoengine/document.py", line 298, in save
    signals.post_save.send(self.__class__, document=self, created=created)
  File "/Users/olive/.virtualenvs/1flow/lib/python2.7/site-packages/blinker/base.py", line 267, in send
    for receiver in self.receivers_for(sender)]
  File "/Users/olive/sources/1flow/oneflow/core/models/nonrel.py", line 1546, in signal_post_save_handler
    document.post_save_task.delay()
  File "/Users/olive/.virtualenvs/1flow/lib/python2.7/site-packages/celery/app/task.py", line 358, in delay
    return self.apply_async(args, kwargs)
  File "/Users/olive/.virtualenvs/1flow/lib/python2.7/site-packages/celery/app/task.py", line 463, in apply_async
    link=link, link_error=link_error, **options)
  File "/Users/olive/.virtualenvs/1flow/lib/python2.7/site-packages/celery/app/task.py", line 638, in apply
    request=request, propagate=throw)
  File "/Users/olive/.virtualenvs/1flow/lib/python2.7/site-packages/celery/task/trace.py", line 331, in eager_trace_task
    uuid, args, kwargs, request)
  File "/Users/olive/.virtualenvs/1flow/lib/python2.7/site-packages/celery/task/trace.py", line 233, in trace_task
    R = retval = fun(*args, **kwargs)
TypeError: post_save_task() takes exactly 1 argument (2 given)

In my tests, I have:

@override_settings(STATICFILES_STORAGE=
                   'pipeline.storage.NonPackagingPipelineStorage',
                   CELERY_EAGER_PROPAGATES_EXCEPTIONS=True,
                   CELERY_ALWAYS_EAGER=True,
                   BROKER_BACKEND='memory',)
class ArticleDuplicateTest(TestCase):


    def setUp(self):

        Article.drop_collection()

        self.article1 = Article(title='test1',
                                url='http://test.com/test1').save()

The crash doesn't happen in "normal" conditions: if I remove the settings override, the post_save_task() runs without problem, but not in the right database (obviously because of production broker, different mongoengine connection, etc).

The task-related code:

from mongoengine import Document, ValidationError
from celery.contrib.methods import task as celery_task_method

class Article(Document):

    #[…]

    @classmethod
    def signal_post_save_handler(cls, sender, document, **kwargs):
        if kwargs.get('created', False):
            document.post_save_task.delay()

    @celery_task_method(name='Article.post_save', queue='high')
    def post_save_task(self):

        self.slug = slugify(self.title)
        self.save()

Any hint/advise ?

[rafaponieman]

Reporting occurrence of the same issue. On a normal request, running celery with CELERY_ALWAYS_EAGER=False it works perfectly, however, when I run tasks in tests (with CELERY_ALWAYS_EAGER=True), it passes the class (method's owner) as a parameter to the method apart from passing it to the self argument (it adds an extra arg).

[Karmak23]

I won't be able to test because now all my tasks are regular tasks, for production-ready reasons I needed to make tests work. Next time I need to develop tasks i will retry using tasks as methods :-)

But that's very good news !! Thanks a lot for this patch.

[rafaponieman]

Same case here, we resorted to a normal function task that calls the class method, this way it works fine. Next time we'll try it. Thanks a lot!

[ask]

The task methods are very experimental so I would expect you to find more bugs :)