Clear old Triggers when Triggerer is not running #34906
Description
Apache Airflow version
main (development)
What happened
When a deferrable operator or sensor is run without a Triggerer process, the task gets stuck in the deferred state, and will eventually fail. A banner will show up in the home page saying that the Triggerer is not running. There is no way to remove this message. In the Triggers menu, the trigger that activated is listed there, and there is no way to remove that Trigger from that list.
What you think should happen instead
If the trigger fails, the trigger should be removed from the Trigger menu, and the message should go away.
How to reproduce
The bug occurs when no Triggerer is running.
In order to reproduce,
- Run any DAG that uses a deferrable operator or sensor.
- Allow the task to reach the deferred state.
- Allow the task to fail on its own (i.e. timeout), or mark it as success or failure.
A message will show up on the DAGs page that the Triggerer is dead. This message does not go away
The triggerer does not appear to be running. Last heartbeat was received 6 minutes ago.
Triggers will not run, and any deferred operator will remain deferred until it times out and fails.
A Trigger will show up in the Triggers menu.
Operating System
ubuntu
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
Using breeze for testing
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct