New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

celery 4.1 ETA time is not correct #4221

Closed
fanjindong opened this Issue Aug 22, 2017 · 8 comments

Comments

Projects
None yet
7 participants
@fanjindong

fanjindong commented Aug 22, 2017

this demo.py

from celery import Celery, Task, exceptions, platforms
from connection import ConnectionTask
from datetime import timedelta
import conf

demo = Celery('demo', broker=conf.MQ_BROKER)

demo.conf.update(
    task_serializer='json',
    accept_content=['json'],
    result_serializer='json',
    timezone='Asia/Shanghai',
    worker_max_tasks_per_child=200,
    worker_disable_rate_limits=True,
    task_create_missing_queues=True,
    beat_schedule={
        'demo-add': {
            'task': 'demo.add',
            'schedule': timedelta(seconds=15),
            'options': {'queue': conf.MQ_DEMO},
            'args': [2, 8],
        },
    }
)


@demo.task(base=ConnectionTask, bind=True, ignore_result=True, max_retries=5)
def add(self, x, y):
    if x==1:
        self.retry(countdown=60)
    return x + y

err behavior

[2017-08-22 16:00:53,025: INFO/MainProcess] Received task: demo.add[5359c683-28c5-4265-bc77-a02807e7827a]  
[2017-08-22 16:00:53,028: INFO/ForkPoolWorker-2] Task demo.add[5359c683-28c5-4265-bc77-a02807e7827a] retry: Retry in 60s
[2017-08-22 16:00:53,035: INFO/MainProcess] Received task: demo.add[5359c683-28c5-4265-bc77-a02807e7827a]  ETA:[2017-08-22 08:01:53.026906+08:06]

look: ETA:[...] time is not correct,but celery 4.0.2 this demo.py is correct

@djluo

This comment has been minimized.

djluo commented Aug 23, 2017

因为时区的关系,不能直接用countdown=60,可以改用eta这种绝对时间。
就是用当前时间点再加60秒的方式,然后用pytz模块带上时区信息。
Sorry that there is no reply in English

import pytz
import datetime
def EtaWithTZ(value):
TZ = pytz.timezone(CELERY_TIMEZONE)
ETA = datetime.datetime.strptime(value, "%Y-%m-%d %H:%M:%S")
return TZ.localize(ETA)

...

eta=EtaWithTZ("2017-08-23 10:30:00")
self.retry(eta=eta)

@ldsink

This comment has been minimized.

ldsink commented Aug 23, 2017

#4173 pull request maybe fix this problem.

@fanjindong

This comment has been minimized.

fanjindong commented Aug 23, 2017

@djluo 木有事,反正我看得懂。

@fanjindong

This comment has been minimized.

fanjindong commented Aug 23, 2017

@ldsink will be fix in celery4.2 ?

@vst

This comment has been minimized.

vst commented Sep 2, 2017

Confirming that the pull request fixed the issue which I started experiencing after moving from v4.0.2 to v4.1.0.

Problem was that the cron job was triggering at the specified time but in UTC time instead of the given timezone. I suspect that this was due to my USE_TZ = False (I need to keep it as is unlike the documentation recommends).

For reference, my settings are:

## Django settings:
USE_TZ = False
TIME_ZONE = '<Some Timezone>'

## Celery Application Settings:
CeleryApplication.conf.update(
    timezone=settings.TIME_ZONE
)
@georgepsarakis

This comment has been minimized.

Member

georgepsarakis commented Sep 2, 2017

@vst thanks for confirming!

@mbarchein

This comment has been minimized.

mbarchein commented Sep 19, 2017

I think this issue also affects retry(countdown=N). With no code changes in my Django project, with Celery 4.0.2 retry(countdown=5) works as expected. Celery 4.1.0 requeues the task but it doesn't get executed in the desired countdown seconds, but several minutes later.

@thedrow

This comment has been minimized.

Member

thedrow commented Sep 19, 2017

Since this is fixed in #4173 I'm going to close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment