Replies: 1 comment
-
You can contribute to the discussion about future datasets. We are currently rethinking Airlfow in the context of Airlfow 3 - and this discussion https://lists.apache.org/thread/k0rxmvwrkghodpfvq1qzbr6tjoptcn1h particularly is about future of datasets, so likely your contribution to the dicussion there is best way to get your points passed. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Prior to datasets, ExternalTaskSensor was the common way to do cross dag dependencies
The examples I've found so far for datasets all involve replacing schedule at the dag level. However, to have drop in replacement for ExternalTaskSensor, I'd ideally want an async sensor that triggers off dataset.
The alternative would be to split up the dag into many dags which makes things harder to read / logic harder to follow
Beta Was this translation helpful? Give feedback.
All reactions