celery.execute.send_task should respect CELERY_ALWAYS_EAGER

[kmike]

Trying to figure out why CELERY_ALWAYS_EAGER doesn't work I found that celery.app.base.BaseApp.send_task (and thus celery.execute.send_task) doesn't respect CELERY_ALWAYS_EAGER.

I think CELERY_ALWAYS_EAGER should be respected both for direct task invocations (via task.delay) and for indirect invocations (via execute.send_task).

[mher]

send_task can be used when there is no access to task class. In that case it is not possible to execute the task locally when CELERY_ALWAYS_EAGER is enabled.

[ask]

Yeah, that would be weird, as the task may not be registered. It should be fairly easy to add this behavior to your own code though.

[kmike]

This make sense. Maybe docs are worth changing then? This behavior is not clear from the docs:

http://celery.readthedocs.org/en/latest/configuration.html#celery-always-eager

CELERY_ALWAYS_EAGER
If this is True, all tasks will be executed locally by blocking until the task returns.

http://celery.readthedocs.org/en/latest/userguide/executing.html#basics

You can also execute a task by name using send_task(), if you don’t have access to the task class:

[litchfield]

This has bitten me too- any reason we can't pull the task class itself from the registry and run it so we get some consistent behaviour for CELERY_ALWAYS_EAGER?

Otherwise we should call it CELERY_MOSTLY_EAGER ;-)

[ask]

Closed by 703cba6

unittests should mock the call to send_task, here is an example using the mock library:

from mock import Mock

def test_task_sent(self):
    from celery import current_app as celery 
    prev = celery.send_task
    celery.send_task = Mock()
    try:
          some_code_sending_task()
          self.assertTrue(celery.send_task.called)
          # or:
          ## celery.send_task.assert_called_with(*expected_args, **expected_kwargs)
    finally:
        celery.send_task = sent

[litchfield]

great! the issue isn't resolved though, so why are we closing it? it either needs to be fixed, or the inconsistency documented.

[Ignas]

I think 703cba6 is the way it got documented.

[ask]

There is a semantic difference between task.apply_async and send_task; i.e It is immediately obvious what eager task.apply_async would do, but not for send_task. Calling tasks that happen to be registered then would be like defining the value of a division by zero. If you want this behavior you could easily write your own send_task that does exactly that.

I don't mind this being documented with the CELERY_ALWAYS_EAGER setting, would you like to write a doc-patch?

[davidmarble]

When testing django apps, I do some monkey patching to get the same result as CELERY_ALWAYS_EAGER = True:

In settings.py:

if 'test' in sys.argv:
    # Monkey patch send_task to use a sync version
    from celery import execute
    from tests.celery import send_task as send_task_test
    execute.send_task = send_task_test

Then in tests/celery.py I have (could use some feedback on this):

def send_task(name, args=None, kwargs=None, countdown=None,
            eta=None, task_id=None, publisher=None, connection=None,
            result_cls=None, expires=None, queues=None, **options):
    module = name[:name.rfind('.')]
    func_name = name[name.rfind('.')+1:]
    exec 'from %s import %s as func' % (module, func_name)
    if args and kwargs:
        result = func(*args, **kwargs)
    elif args:
        result = func(*args)
    elif kwargs:
        result = func(**kwargs)
    return result

[ask]

@davidmarble

Thanks for posting your solution, you could shorten it a bit:

from celery import current_app

def send_task(name, args=(), kwargs={}, **opts):
    task = current_app.tasks[name]
    return task.apply(args, kwargs, **opts)

if 'test' in sys.argv:
    current_app.send_task = send_task

This will return a EagerResult though, like with ALWAYS_EAGER,
if you want the result directly you can use return task(*args, **kwargs)

[BeOleg]

Proper solution?

[skinderis]

Would be great to have proper solution. Is there any?

[BeOleg]

@ask ?

[auvipy]

ask do not maintain celery anymore.