deferred tasks get kill during heartbeat callback in some rare cases #40435
Labels
affected_version:2.8
Issues Reported for 2.8
area:Scheduler
Scheduler or dag parsing Issues
good first issue
kind:bug
This is a clearly a bug
Apache Airflow version
Other Airflow 2 version (please specify below)
If "Other Airflow 2 version" selected, which one?
2.8.1 (MWAA)
What happened?
My airflow deferred (AWS) BatchOperator occasionally fail. When this happens I can see this:
I actually think it's an oversight in the code for local_task_job_runner. This line should check for state being RUNNING or DEFERRED.
What you think should happen instead?
No response
How to reproduce
This is hard to reproduce. The issue is very transient.
Operating System
MWAA
Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.16.0
Deployment
Official Apache Airflow Helm Chart
Deployment details
MWAA
Anything else?
I would say it happens one out of 100 run.
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: