Cronjob is run every time runcrons is called

[sulami]

We just noticed some strange behaviour on our staging server after updating from django-cron 0.4.6 to 0.5.0. One of our jobs looks like this:

class AnalyticsReportJob(CronJobBase):
    RUN_EVERY_MINS = 60 * 24

    schedule = Schedule(run_every_mins=RUN_EVERY_MINS)
    code = 'analytics.report'

    def do(self):
        do_stuff()

It's supposed to report on gathered analytics data once a day, basically just collecting and sending out some emails. Real cron is calling runcrons every minute. After experiencing some heavy load and unexpected behaviour, we could confirm that do_stuff() is actually executed every minute, leading to several critical issues including zombie processes.

[DanEEStar]

I have the same problem.
A job which should run once a day, configured via run_at_times seems to run every minute.
Python processes are started every minute and get never killed.

I didn't use this library before version 0.5.0 so I cannot say if it has something to do with the upgrade...

Even worse, this behavior was only seen on the production environment. On out staging environment it seemed to work. The staging and prod environment use the same libraries and dependencies, so this is really strange...

[craiglabenz]

Have you verified that as soon as a job starts, an entry appears in the log table?

[DanEEStar]

When I was viewing the log table, after about 10 minutes of the deploy, there were multiple entries of the same job in there. Each one had a start time of 1 minute apart.
Then I killed all python processes. I also delete all entries in the job table.

Then when I restarted the application the job got started multiple times again, as I could verify via the ps command on the server. But it did not create new entries in the job table. The job table remained empty this time...

Sorry that I cannot be more specific. But the behaviour forced me to make some emergency adjustments after the deploy.
I can try it again, when I have more time.

[tab-cmd]

@DanEEStar and @sulami the only difference I could see between the two envs and versions would be in how we get the time now:

def get_current_time():
now = utc_now()
return now if is_naive(now) else localtime(now)

What is your timezone and is it USE_TZ? I think I know how to fix, but want to make sure. Could you locally be working in different timezones between env settings? Thanks for the info

[sulami]

I don't have access to the environment in question anymore, but we did use USE_TZ, and I'm pretty sure the servers were in Ireland (AWS).

[pvojnov]

Hi, i have the same issue but the provided fix didn't solve it.

By looking at the code it seems that the CronJobLog record doesn't get inserted to the database until the job finishes.
That way the query in CronJobManager.should_run_now wont find the job that is already running and start the job again before the first one finishes.

I've made a quick fix to CronJobManager.run by adding:
self.make_log('Job Running!!', success=True)
Just before the line:
self.msg = self.cron_job.do()

And it works!! :)