Repository navigation
Allow tasks downstream of setup tasks to have trigger_rule other than ALL_SUCCESS
#48992
Replies: 2 comments
|
Same issue as #48451 -> converting to a discussion. Setup and teardown are designed in the way that they MUST succed. If you want to do conditional behaviour of those, you have to implement "ifs" in your taskflow code that is used for setup/teardown and avoid doing any job, but still succeed. This is far simpler and straightforward and follows the - clearly documented semantics of setup/teardown tasks. The setup tasks mus be "green" (no matter what they do or don't do) in order to exacute whatever depends on setup. This is the basic assumption here. |
|
cc: @dstandish I 'd love your opinion here, but I do not expect my understanding on how setup should behave is wrong, and I believe we absolutely do not want to make it more complex by introducing completely new idea of "skippable" setup and teardown. |
Uh oh!
There was an error while loading. Please reload this page.
Description
I have a DAG where I want to leverage
setupandteardowntasks. However, thesetuptask is optional - I want it to be skipped if the value of an Airflow variable isFalse. If it's skipped, I still want its downstream task to execute - In order to do this, the downstream task must havetrigger_rule=NONE_FAILED.The implicit
ALL_SUCCESScontstraint prevents me from doing this, causing a DAG parse failure:Use case/motivation
On our Airflow instance deployed to our server, we want to drop a database's replication slot in the
setuptask, then we have several tasks that insert data into the database, then we add back the replication slot in theteardowntask.When we run this DAG locally, there is no replication slot, so I want to show the
setup/teardowntask areskippedbecause they won't actually do anything.Right now, I can't set the tasks downstream of the
setuptask to havetrigger_rule=NONE_FAILEDbecause they are required to havetrigger_rule=ALL_SUCCESS.Related issues
No response
Are you willing to submit a PR?
Code of Conduct
All reactions