OverflowError('date value out of range',) when CELERY_TIMEZONE has negative UTC offset

[SpotlightKid]

This happens when a task is written into Redis and then on the next tick, when the task is loaded from Redis and .is_due() is called on the task:

{ "pid": 31792, "message": "Selecting tasks", "python_module": "redbeat.schedulers", "level": "DEBUG", "timestamp": "2016-08-12 09:24:11,855" }
{ "pid": 31792, "message": "Loading 1 tasks", "python_module": "redbeat.schedulers", "level": "INFO", "timestamp": "2016-08-12 09:24:11,856" }
{ "pid": 31792, "message": "Processing tasks", "python_module": "redbeat.schedulers", "level": "DEBUG", "timestamp": "2016-08-12 09:24:11,857" }
{ "pid": 31792, "message": "beat: Releasing Lock", "python_module": "redbeat.schedulers", "level": "DEBUG", "timestamp": "2016-08-12 09:24:11,870" }
{ "pid": 31792, "message": "beat raised exception <type 'exceptions.OverflowError'>: OverflowError('date value out of range',)", "python_module": "celery.beat", "level": "CRITICAL", "timesta
mp": "2016-08-12 09:24:11,872" }
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/apps/beat.py", line 112, in start_scheduler
    beat.start()
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/beat.py", line 463, in start
    interval = self.scheduler.tick()
  File "/usr/local/lib/python2.7/dist-packages/redbeat/schedulers.py", line 264, in tick
    return super(RedBeatScheduler, self).tick(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/beat.py", line 221, in tick
    next_time_to_run = self.maybe_due(entry, self.publisher)
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/beat.py", line 199, in maybe_due
    is_due, next_time_to_run = entry.is_due()
  File "/usr/local/lib/python2.7/dist-packages/redbeat/schedulers.py", line 175, in is_due
    return super(RedBeatSchedulerEntry, self).is_due()
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/beat.py", line 129, in is_due
    return self.schedule.is_due(self.last_run_at)
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/schedules.py", line 117, in is_due
    last_run_at = self.maybe_make_aware(last_run_at)
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/schedules.py", line 126, in maybe_make_aware
    return maybe_make_aware(dt, self.tz)
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/utils/timeutils.py", line 313, in maybe_make_aware
    dt, timezone.utc if tz is None else timezone.tz_or_local(tz),
  File "/usr/local/lib/python2.7/dist-packages/celery-3.1.17-py2.7.egg/celery/utils/timeutils.py", line 289, in localize
    dt = dt.astimezone(tz)
  File "/usr/local/lib/python2.7/dist-packages/pytz/tzinfo.py", line 187, in fromutc
    return (dt + inf[0]).replace(tzinfo=self._tzinfos[inf])
OverflowError: date value out of range

Analysis:

• Happens only when CELERY_TIMEZONE is set to a timezone with a negative UTC offset (e.g. "America/Chicago" in our case
• Task has never run before

When Redbeat loads a task from Redis (redBeatdSchedulerEntry.from_key) it tries to get the last_run_at attribute value from the meta dict in the Redis hashset of the task. When the task has never run, this is empty, so it sets last_run_at to datetime.min. Then in is_due (down the call chain), last_run_at is passed to celery.utils.timeutils.maybe_make_aware, which tries to convert the timestamp into a timezone-aware datetime object, using the scheduler timezone. This lead to the OverflowError, because you can't apply a negative offset to datetime.min.

Proposed solution:

If last_run_at is not set in the task meta info, set it to None and in RedBeatSchedulerEntry.is_due, if last_run_at is None, pass datetime.dtatime(datetime.MINYEAR, 1, 1, tzinfo=self.schedule.tz) to self.schedule.is_due. In RedBeatSchedulerEntry.is_at if last_run_at is None, return self._default_now().

I'll prepare a PR for this.