Update the docs to make it clear tasks must be published from within the Django context

[MikeSandford]

I'm just getting started with carrot on a new project after having fairly extensive experience with celery. A lot of the "why carrot?" explanations resonated with me, so thanks very much for making this!

I created a carrot_test.py file in the root of my django project, next to manage.py and then tried to run it. When I did, I saw this:

(env) sandford@mjolnir ~/personal/website $ python carrot_test.py 
Traceback (most recent call last):
  File "carrot_test.py", line 12, in <module>
    from carrot.utilities import publish_message
  File "/home/sandford/personal/website/env/lib/python3.5/site-packages/carrot/utilities.py", line 15, in <module>
    from carrot.models import ScheduledTask
  File "/home/sandford/personal/website/env/lib/python3.5/site-packages/carrot/models.py", line 18, in <module>
    class MessageLog(models.Model):
  File "/home/sandford/personal/website/env/lib/python3.5/site-packages/django/db/models/base.py", line 100, in __new__
    app_config = apps.get_containing_app_config(module)
  File "/home/sandford/personal/website/env/lib/python3.5/site-packages/django/apps/registry.py", line 244, in get_containing_app_config
    self.check_apps_ready()
  File "/home/sandford/personal/website/env/lib/python3.5/site-packages/django/apps/registry.py", line 127, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
(env) sandford@mjolnir ~/personal/website $ 

Is this the end of the world? Not at all, I've seen similar when trying to run things that need django on the path, so I dug around on the 'net and found this stanza:

import os
from django.core.wsgi import get_wsgi_application
os.environ['DJANGO_SETTINGS_MODULE'] = 'mail.settings'
application = get_wsgi_application()

Then when I run it, it seems to work! This is great!

(env) sandford@mjolnir ~/personal/website $ python carrot_test.py 
(env) sandford@mjolnir ~/personal/website $ 

But when I check my carrot.log I see this:

default-consumer-1 2018-05-18 12:57:27,390 INFO:: Consuming task __main__.my_task, ID=e2517842-d3a3-49d4-9844-708ea082a415
default-consumer-1 2018-05-18 12:57:27,390 ERROR:: Task __main__.my_task failed due to the following exception: module '__main__' has no attribute 'my_task'

It seems like there's some confusion about where/how carrot should be looking to find the function itself.

I was able to solve my problem by putting the task into a module that was already in the django path, importing it in carrot_test.py and then executing and that worked. But it was a bit of a challenge to figure out what would work and what wouldn't. Initially the only way I figured out how to get it to work was by firing up a python shell and importing carrot_test and I just happened to see that got the task run successfully without erroring. It was a confusing first hour until I understood what was going on.

If you're triggering jobs from a view function and your tasks are already in packages that are on the django path I don't think you'll run into any of these problems. But when getting started for the very first time I at least tend to try the simplest thing like firing up a single-purpose file.

[chris104957]

I'll see if i can make the docs a bit clearer, but in general the modules containing your tasks should be kept in an app directory. What's in your carrot_test file exactly?

[MikeSandford]

Hey, thanks for getting back to me! Here's my file including the comments so that you can see what I tried:

#import django
#django.setup()

import os
from django.core.wsgi import get_wsgi_application
os.environ['DJANGO_SETTINGS_MODULE'] = 'mail.settings'
application = get_wsgi_application()


from carrot.utilities import publish_message
#from jobs.jobs import my_task

def my_task(**kwargs):
    return 'hello world'

#if __name__ == "__main__":
publish_message(my_task, hello=True)

As you can see I did figure out that I needed to make the function available to django and I was able to get things working. Also worth noting that if I uncomment the name==main bit I don't get the task scheduled simply by importing the file anymore.

I was thinking about making a PR for a management command for something like python manage.py carrot_queue package.module.function to manually add a task to get run, the implication hopefully being that if you don't have the task somehow available to django it's going to be hard for the thing to work and maybe the code which does the importing can complain so you get proactive feedback. Would you be interested in something like that?

[chris104957]

Sure, that could work. You could leverage the publish_task method in carrot.utilities which validates import paths before publishing a task. Would you have the same method start the consumer as well, or just work on the basis that it would already be running?

[MikeSandford]

Yeah I'll take a stab at it and see how I do. Thanks for the feedback!

[chris104957]

Docs have been updated for 1.2.0