You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Airflow scheduler is not sending heartbeat - not spawning subprocesses (DagFileProcessManager, workers,...) when running in daemon mode
#49109
I first activated my python env, exported AIRFLOW_HOME variable, then started airflow webserver in daemon mode normally with: airflow webserver -p 8080 -D.
When I started airflow scheduler with daemon mode: airflow scheduler -D, the webserver UI informed me that there is no scheduler heartbeat and I can even not manually trigger dags. However, when I run it in the foreground, every thing runs smoothly. (I make sure that i killed processes of old scheduler process including 1 master process and the command process)
After researching about scheduler's heartbeat, I was provided that scheduler periodically write timestamp "heartbeat" to database. Moreover, I saw that there was no DagFileProcessManager process and other executor processes. I could not come into conclusion whether the scheduler's process is stucked in kinda a loop or got failed silently before spawning subprocesses.
Another thing is that, whenever I tried to shut down foreground scheduler making some tests, it noticed that it is waiting for running tasks to finish though i turned off all my dags
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Prerequisites:
Problem
AIRFLOW_HOMEvariable, then started airflow webserver in daemon mode normally with:airflow webserver -p 8080 -D.airflow scheduler -D, the webserver UI informed me that there is no scheduler heartbeat and I can even not manually trigger dags. However, when I run it in the foreground, every thing runs smoothly. (I make sure that i killed processes of old scheduler process including 1 master process and the command process)Can someone explain what is going on?
All reactions