job expiration not working

[aydas]

Hi ,

I still have problem with WOOEY_JOB_EXPIRATION parameter. I set this to:

user_job_expire=timedelta(hours=1)
anonymous_job_expire=timedelta(minutes=10)

WOOEY_JOB_EXPIRATION = {'user':user_job_expire,'anonymous':anonymous_job_expire}

But no success.
Please help me to fix

Aidas

[zz138960]

task:cleanup-old-jobs run at midnight each day,so the results will not be verified until the next day
celery_app.conf.beat_schedule.update({
'cleanup-old-jobs': {
'task': 'wooey.tasks.cleanup_wooey_jobs',
'schedule': crontab(hour=0, minute=0), # cleanup at midnight each day
},
'cleanup-dead-jobs': {
'task': 'wooey.tasks.cleanup_dead_jobs',
'schedule': crontab(minute='*/10'), # run every 6 minutes
}
})

[aydas]

Hi,
Thank you for answer.
Problem is that system still keep all competed jobs from beginning (more then month) even after I set WOOEY_JOB_EXPIRATION parameter to 1 hour (in timedelta) for user.
So I guess problem is in parameter format or in celery side or somewhere else.
Please help me to understand how to fix this.

[zz138960]

Please confirm whether the beat command works normally，Has your celery received cleanup_wooey_jobs task or cleanup_dead_jobs task?

[Chris7]

Keep in mind if you are running windows, celery beat is not really supported. I've found some ways to get it running, but the reliability probably varies and is not guaranteed between versions.
https://wooey.readthedocs.io/en/latest/running_wooey.html#through-two-separate-processes

``
On Windows, the --beat option may not be supported and the eventlet pool will need to be specified. This looks like:

celery -A your_project_name worker --pool=eventlet -l info

[aydas]

wooey is running on ubuntu.
python jobs is running and executed as excepted, Using rabbitmq.
The problem is with scheduled tasks like cleanup-old-jobs only.
Is there any way to check is cleanup-old-jobs started and completed successfully?

[Chris7]

Scheduled tasks should be getting recorded in the celery output. Could you provide how you are running wooey and celery?

[aydas]

I setup production server on digitalocean droplet. As broker using rabbitmq.
And set supervisor service.
Celery see tasks:
[tasks]
. inovega.wooey_celery_app.debug_task
. wooey.tasks.cleanup_dead_jobs
. wooey.tasks.cleanup_wooey_jobs
. wooey.tasks.submit_script

But log shows wooey.tasks.submit_script tasks only, no other tasks:

[2020-09-11 08:42:02,496: INFO/MainProcess] Received task: wooey.tasks.submit_script[19b1bfd1-fefc-4971-bb84-ec60245b2efa]
[2020-09-11 08:42:03,443: INFO/ForkPoolWorker-1] Task wooey.tasks.submit_script[19b1bfd1-fefc-4971-bb84-ec60245b2efa] succeeded

Any ideas, that is wrong?

Aidas

[Chris7]

How are you invoking hte celery command? I don't see anything indicating the beat option is being used.

[aydas]

Hi, Thank you for answer. Celery is connected under supervisor service, in supervisor config file is inserted line: celery -A appname worker -l info Can this service configuration cause inactivity of cleanup-old-jobs? Aidas 2020-10-04, sk, 17:12 Chris Mitchell <notifications@github.com> rašė:

…

How are you invoking hte celery command? I don't see anything indicating the beat option is being used. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#328 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGGQ2NVAPV2JOOWWHWHUMLSJB7DBANCNFSM4QMEOEUQ> .

[Chris7]

Hi @aydas ,

You need to add the --beat command to your celery worker. See https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#starting-the-scheduler. It should be celery -A appname worker -l info --beat

[aydas]

Thank you, I will try to modify and test again Aidas 2020-10-05, pr, 15:46 Chris Mitchell <notifications@github.com> rašė:

…

Hi @aydas <https://github.com/aydas> , You need to add the --beat command to your celery worker. See https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#starting-the-scheduler. It should be celery -A appname worker -l info --beat — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#328 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGGQ2LWILGP2GMKHKX5GRTSJG54BANCNFSM4QMEOEUQ> .

[aydas]

after changes to celery -A appname worker -l info --beat wooey jobs goes to status waiting... after changes to celery -A appname worker -l info (without --beat) and restarting supervisor tasks still on waiting? This rabbitqm or celery problem? Aidas 2020-10-05, pr, 15:49 Aidas Kaubrys <aidas.kaubrys@gmail.com> rašė:

…

Thank you, I will try to modify and test again Aidas 2020-10-05, pr, 15:46 Chris Mitchell ***@***.***> rašė: > Hi @aydas <https://github.com/aydas> , > > You need to add the --beat command to your celery worker. See > https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#starting-the-scheduler. > It should be celery -A appname worker -l info --beat > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#328 (comment)>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/ADGGQ2LWILGP2GMKHKX5GRTSJG54BANCNFSM4QMEOEUQ> > . >

[aydas]

Hi Chris, After change supervisor configuration and inserting beat command, scheduled cleanup tasks starts working but scripts stopped with status waiting, so I think problem is with celery configuration. Aidas 2020-10-05, pr, 15:46 Chris Mitchell <notifications@github.com> rašė:

…

Hi @aydas <https://github.com/aydas> , You need to add the --beat command to your celery worker. See https://docs.celeryproject.org/en/stable/userguide/periodic-tasks.html#starting-the-scheduler. It should be celery -A appname worker -l info --beat — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#328 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGGQ2LWILGP2GMKHKX5GRTSJG54BANCNFSM4QMEOEUQ> .

[Chris7]

It seems to be. The management panel of your broker may be a good place to look to see if tasks are piling up there or being sent at all.

[aydas]

OK. thank you. for support. I would like to iadd LT language into wooey, can you help me with some advise how-to-do, is it standart Django funkcionality or is something non standart Aidas 2020-10-06, an, 15:43 Chris Mitchell <notifications@github.com> rašė:

…

It seems to be. The management panel of your broker may be a good place to look to see if tasks are piling up there or being sent at all. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#328 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGGQ2IG2ZNJMOBDRWNDF3LSJMGILANCNFSM4QMEOEUQ> .

[Chris7]

Hi @aydas ,

I love translations :). The translation system is part of the standard Django package. To add a new translation, you'd do this:

Run django-admin makemessages --locale the_locale_to_add from the wooey directory (where git clone drops you). This will create a directory under wooey/locale/the_locale_name with a .po file. This file is where you enter the human readable text. Following translations, you run django-admin compilemessages --locale the_locale_to_add.

How to add a new translations I think would be a great thing to add to the docs as well.

[aydas]

I will try, Thank you 2020-10-13, an, 16:44 Chris Mitchell <notifications@github.com> rašė:

…

Hi @aydas <https://github.com/aydas> , I love translations :). The translation system is part of the standard Django package. To add a new translation, you'd do this: Run django-admin makemessages --locale the_locale_to_add from the wooey directory (where git clone drops you). This will create a directory under wooey/locale/the_locale_name with a .po file. This file is where you enter the human readable text. Following translations, you run django-admin compilemessages --locale the_locale_to_add. How to add a new translations I think would be a great thing to add to the docs as well. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#328 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGGQ2P5UGXWVZXQIWCNHGLSKRKTTANCNFSM4QMEOEUQ> .