You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm looking at building an ETL pipeline (p1) in Airflow which triggers a different DAG (p2) using the TriggerDagRunOperator but I want p1 to never fail based off the result of p2. As the failed_state argument will default to ['failed'] if passed an empty array I was wondering if there is a way to pass an argument which would allow no failed states. In practice, this works by passing failed_states=['dummy'] into the TriggerDagRunOperator but this fails the internal_unit_testing.assert_has_valid_dag() tests I run. Below is what I want to be able to run but defaults to failed_state=['failed']
Is it possible for the failed_states check in this operator to be more specific about whether it is the default None or if it is actually being passed an empty array?
Currently the code just checks if the failed_states arg is truthy which an empty array fails.
if failed_states:
self.failed_states = [DagRunState(s) for s in failed_states]
else:
self.failed_states = [DagRunState.FAILED]
but I feel this could be fixed with:
if failed_states is None:
self.failed_states = [DagRunState.FAILED]
else:
self.failed_states = [DagRunState(s) for s in failed_states]
If this behaviour is possible without a code change, that would be awesome too.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I'm looking at building an ETL pipeline (p1) in Airflow which triggers a different DAG (p2) using the TriggerDagRunOperator but I want p1 to never fail based off the result of p2. As the failed_state argument will default to ['failed'] if passed an empty array I was wondering if there is a way to pass an argument which would allow no failed states. In practice, this works by passing failed_states=['dummy'] into the TriggerDagRunOperator but this fails the internal_unit_testing.assert_has_valid_dag() tests I run. Below is what I want to be able to run but defaults to failed_state=['failed']
Is it possible for the failed_states check in this operator to be more specific about whether it is the default None or if it is actually being passed an empty array?
Currently the code just checks if the failed_states arg is truthy which an empty array fails.
but I feel this could be fixed with:
If this behaviour is possible without a code change, that would be awesome too.
Beta Was this translation helpful? Give feedback.
All reactions