celery 4.1 ETA time is not correct

[fanjindong]

this demo.py

from celery import Celery, Task, exceptions, platforms
from connection import ConnectionTask
from datetime import timedelta
import conf

demo = Celery('demo', broker=conf.MQ_BROKER)

demo.conf.update(
    task_serializer='json',
    accept_content=['json'],
    result_serializer='json',
    timezone='Asia/Shanghai',
    worker_max_tasks_per_child=200,
    worker_disable_rate_limits=True,
    task_create_missing_queues=True,
    beat_schedule={
        'demo-add': {
            'task': 'demo.add',
            'schedule': timedelta(seconds=15),
            'options': {'queue': conf.MQ_DEMO},
            'args': [2, 8],
        },
    }
)


@demo.task(base=ConnectionTask, bind=True, ignore_result=True, max_retries=5)
def add(self, x, y):
    if x==1:
        self.retry(countdown=60)
    return x + y

err behavior

[2017-08-22 16:00:53,025: INFO/MainProcess] Received task: demo.add[5359c683-28c5-4265-bc77-a02807e7827a]  
[2017-08-22 16:00:53,028: INFO/ForkPoolWorker-2] Task demo.add[5359c683-28c5-4265-bc77-a02807e7827a] retry: Retry in 60s
[2017-08-22 16:00:53,035: INFO/MainProcess] Received task: demo.add[5359c683-28c5-4265-bc77-a02807e7827a]  ETA:[2017-08-22 08:01:53.026906+08:06]

look: ETA:[...] time is not correct,but celery 4.0.2 this demo.py is correct

[djluo]

因为时区的关系，不能直接用countdown=60，可以改用eta这种绝对时间。
就是用当前时间点再加60秒的方式，然后用pytz模块带上时区信息。
Sorry that there is no reply in English

import pytz
import datetime
def EtaWithTZ(value):
TZ = pytz.timezone(CELERY_TIMEZONE)
ETA = datetime.datetime.strptime(value, "%Y-%m-%d %H:%M:%S")
return TZ.localize(ETA)

...

eta=EtaWithTZ("2017-08-23 10:30:00")
self.retry(eta=eta)

[ldsink]

#4173 pull request maybe fix this problem.

[fanjindong]

@djluo 木有事，反正我看得懂。

[fanjindong]

@ldsink will be fix in celery4.2 ？

[vst]

Confirming that the pull request fixed the issue which I started experiencing after moving from v4.0.2 to v4.1.0.

Problem was that the cron job was triggering at the specified time but in UTC time instead of the given timezone. I suspect that this was due to my USE_TZ = False (I need to keep it as is unlike the documentation recommends).

For reference, my settings are:

## Django settings:
USE_TZ = False
TIME_ZONE = '<Some Timezone>'

## Celery Application Settings:
CeleryApplication.conf.update(
    timezone=settings.TIME_ZONE
)

[georgepsarakis]

@vst thanks for confirming!

[mbarchein]

I think this issue also affects retry(countdown=N). With no code changes in my Django project, with Celery 4.0.2 retry(countdown=5) works as expected. Celery 4.1.0 requeues the task but it doesn't get executed in the desired countdown seconds, but several minutes later.

[thedrow]

Since this is fixed in #4173 I'm going to close this issue.