-
Notifications
You must be signed in to change notification settings - Fork 13.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CeleryExecutor gevent/eventlet pools need monkey patching #8023
Comments
Thanks for opening your first issue here! Be sure to follow the issue template! |
@aamangeldi would you like to open a PR with suggested change? 🚀 |
@turbaszek Yup, can do. |
@aamangeldi thank you! |
I experienced the same issue using exactly the same setup (Airflow 1.10.9 with puckel/docker-airflow). @aamangeldi s patch fixes the issue for me. Is there any schedule for bringing this upstream? |
We can push that out in 1.10.11 |
Apache Airflow version:
1.10.9
Kubernetes version (if you are using kubernetes) (use
kubectl version
):Note: the issue is not specific to k8s.
Environment:
Any. I was able to reproduce using CeleryExecutor docker-compose in the puckel repo (code version tagged as 1.10.9).
What happened:
When setting
pool
in the[celery]
section in airflow.cfg toeventlet
orgevent
, task instances get scheduled, queued, picked up by the workers, but not executed.What you expected to happen:
Task instances should be executed. The problem is that the application is not monkey-patched. Celery by default handles monkey-patching but not in all scenarios (e.g. only if Celery is invoked via command line, more info).
Airflow invokes Celery workers in Python via .run(). Unfortunately, this function does not handle monkey patching.
How to reproduce it:
Clone puckel's docker-airflow:
Modify Dockerfile to:
Then:
Set
pool = eventlet
in airflow.cfg (the file will be mounted by docker-compose).Spin up [the CeleryExecutor docker compose](CeleryExecutor docker-compose:
Navigate to http://localhost:8080, and run an example DAG.
Notice that no task ever gets to the running state.
Solution:
Ideally this should be fixed in Celery, but in the meantime it might be good to have a solution here as well. Here is a patch that I applied to solve this (on Airflow 1.10.9):
The text was updated successfully, but these errors were encountered: