-
Notifications
You must be signed in to change notification settings - Fork 13.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TriggerDagRunOperator
not working as expected in defer with wait_for_completion
#38353
Closed
1 of 2 tasks
Labels
affected_version:2.9
area:async-operators
AIP-40: Deferrable ("Async") Operators
area:core
kind:bug
This is a clearly a bug
Comments
TriggerDagRunOperator
not working as expected in defer with wait_for_completion
bug is still present in 2.9.0 |
@raphaelauv anything update? |
@thanhtrung5763 No one proposed a fix yet |
Closed
2 tasks
josh-fell
added a commit
to josh-fell/airflow
that referenced
this issue
May 31, 2024
Closes: apache#38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run.
josh-fell
added a commit
to josh-fell/airflow
that referenced
this issue
Jun 2, 2024
Closes: apache#38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run.
josh-fell
added a commit
to josh-fell/airflow
that referenced
this issue
Jun 2, 2024
Closes: apache#38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run.
josh-fell
added a commit
that referenced
this issue
Jun 2, 2024
…39960) Closes: #38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run.
utkarsharma2
pushed a commit
that referenced
this issue
Jun 4, 2024
…39960) Closes: #38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run. (cherry picked from commit ae648e6)
ephraimbuddy
pushed a commit
that referenced
this issue
Jun 5, 2024
…39960) Closes: #38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run. (cherry picked from commit ae648e6)
ephraimbuddy
pushed a commit
that referenced
this issue
Jun 5, 2024
…39960) Closes: #38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run. (cherry picked from commit ae648e6)
ephraimbuddy
pushed a commit
that referenced
this issue
Jun 5, 2024
…39960) Closes: #38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run. (cherry picked from commit ae648e6)
utkarsharma2
pushed a commit
that referenced
this issue
Jun 5, 2024
…39960) Closes: #38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run. (cherry picked from commit ae648e6)
ephraimbuddy
pushed a commit
that referenced
this issue
Jun 5, 2024
…39960) Closes: #38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run. (cherry picked from commit ae648e6)
fdemiane
pushed a commit
to fdemiane/airflow
that referenced
this issue
Jun 6, 2024
…pache#39960) Closes: apache#38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run.
jannisko
pushed a commit
to jannisko/airflow
that referenced
this issue
Jun 15, 2024
…pache#39960) Closes: apache#38353 When using the TriggerDagRunOperator in `deferrable=True` mode, the DagStateTrigger is being passed the incorrect logical date to poll for. The trigger is using a logical date that is calculated on every execution rather than the logical from either the triggered DAG run or an existing DAG run (if the task is configured to not fail for existing DAG runs). This change corrects the logical date being used by the DagStateTrigger to poll for the triggered (or reset) DAG run.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
affected_version:2.9
area:async-operators
AIP-40: Deferrable ("Async") Operators
area:core
kind:bug
This is a clearly a bug
Apache Airflow version
2.8.3
If "Other Airflow 2 version" selected, which one?
No response
What happened?
using the triggerdagrunoperator with wait_for_completion and deferrable show multiple bugs
for long running dags the triggerdagrunoperator sometimes never finish ( do not detect the end success or fail of triggered dag )
also the triggerdagrunoperator with reset_dag_run never finish if you clear it after a first run
What you think should happen instead?
No response
How to reproduce
triggerdagrunoperator with reset_dag_run never finish if you clear it after a first run
->
trigger the dag_1 wait for end ( it will fail , that's normal ) , clear the task task_1 and the task will stay forever in deferred state
Operating System
ubuntu 22.04
Versions of Apache Airflow Providers
No response
Deployment
Docker-Compose
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: