Carrot service should warn users when process is already running

[MikeSandford]

I'm working on my PR for a django command to queue jobs. It's a little bit ugly trying to support args and kwargs and how to handle types and such.

One thing I've noticed is that the first job I queue when starting carrot up for the first time doesn't get executed and it doesn't matter if the queuing is coming from this management command or a view that also triggers a job.

This might be because I've got some stuck jobs in my queue:

Any thoughts as to what might be going on here?

Here's where I'm at with the management command:

(env) sandford@mjolnir ~/personal/website $ python manage.py carrot_send items.jobs.deskew_envelope "df092c66-61a1-4b8f-bbf0-73d35c2b222a" ""
<items.management.commands.carrot_send.Command object at 0x7f9184cb93c8> () {'job_args': 'df092c66-61a1-4b8f-bbf0-73d35c2b222a', 'traceback': False, 'settings': None, 'no_color': False, 'job_kwargs': '', 'verbosity': 1, 'job_name': 'items.jobs.deskew_envelope', 'pythonpath': None}
['items.jobs.deskew_envelope', ['df092c66-61a1-4b8f-bbf0-73d35c2b222a'], None]
(env) sandford@mjolnir ~/personal/website $ 

[chris104957]

I've seen this reported a couple of times but have been struggling to reliably reproduce this on my side. Please could you let me know:

1. Are you running django-carrot in a docker container?
2. What DB backend you are using for Django?
3. What it says in the carrot log about these tasks?

I have a suspicion what might be causing this, and a potential fix, but I just need to check your definitely seeing the same issue I think you're seeing

[MikeSandford]

1. No, running on my desktop
2. I'm using sqlite since things aren't really settled yet
3. They don't show up at all

default-consumer-1 2018-05-24 00:41:57,469 INFO:: Connecting to amqp://guest:guest@localhost:5672/%2f
default-consumer-1 2018-05-24 00:41:57,474 INFO:: Connection opened
default-consumer-1 2018-05-24 00:41:57,475 INFO:: Channel opened
default-consumer-1 2018-05-24 00:41:57,477 INFO:: Exchange declared
default-consumer-1 2018-05-24 00:41:57,482 INFO:: Queue bound
default-consumer-1 2018-05-24 00:41:57,482 INFO:: Starting consumer default-consumer-1
(env) sandford@mjolnir ~/personal/website $ 

I waited for several seconds after that before I published the message.

But when I look at the failed message log, it's there!

Failure timearrow_upward	Taskarrow_upward	Argumentsarrow_upward	Keyword argumentsarrow_upward	Exceptionarrow_upward
5/23/2018, 7:42:03 PM	items.jobs.deskew_envelope	('df092c66-61a1-4b8f-bbf0-73d35c2b222a',)	{}	invalid literal for int() with base 10: 'df092c66-61a1-4b8f-bbf0-73d35c2b222a'

Then when I run it again, exact same message, somehow it magically works!

5/23/2018, 7:45:01 PM | items.jobs.deskew_envelope | ('df092c66-61a1-4b8f-bbf0-73d35c2b222a',) | {}
-- | -- | -- | --

This to me feels like some kind of weird import error as I've changed the code recently from expecting an int to expecting a UUID, but for some reason it's still failing because it expected an int on the first task execution. I can't figure out why that is.

[MikeSandford]

I should mention that I'm using carrot in non-daemon mode and that after code changes I ctrl-c and then start carrot like this:

(env) sandford@mjolnir ~/personal/website $ python manage.py carrot --logfile carrot.log

[MikeSandford]

I went looking at the code and it became very clear to me that there was no way this could possibly be happening. I thought I had successfully issued a daemon stop, but when I did a ps axf look what was staring me in the face:

18843 ?        Sl    72:15 /usr/lib/firefox/firefox
18917 ?        Sl    21:46  \_ /usr/lib/firefox/plugin-container -greomni /usr/lib/firefox/omni.ja -appomni /usr/lib/firefox/browser/omni.ja -appdir /usr/lib/firefox/browser 18843 true tab
22460 pts/2    Sl     2:01 python3 manage.py carrot --verbosity 1 --logfile carrot.log --loglevel DEBUG
(env) sandford@mjolnir ~/personal/website $ kill 22460

Once I successfully killed that, everything works as expected! Turns out it's my fault :(

So this is unsolicited advice and I'm going to sound super ungrateful, but if carrot had warned me saying "you're running me in non-daemon mode but I see some carrot processes still running in the background, are you sure you want to do this?" I probably would have ctrl-c'd and figured it out.

I'm not sure exactly the best way to do that, of course, other than that it's probably in the management command for non-daemon start. I'll take a look at that while I'm putting this PR together.

[chris104957]

So its not the issue I suspected. I'll see if I can get a warning like that into the next release

[chris104957]

now fixed in 1.2