Replies: 1 comment
-
You can always extend the class and replace the logic that is there with your own logic (and if you find it working well contribute it back as an option for the built-in operator). Airflow is pretty extensible - in the sense, that writing your custom operator is as easy as writing the DAG because it's all Python - so if you need something that is not implemented in the "airflow" class - extending and modifying it and eventually contributing back when it's a generic solution that others might get useful is the standard way to go. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
When using
ExternalTaskSensor
it's doing a lookup on logical_date (datetime)ExternalTaskSensor -> WorkflowTrigger
https://github.com/apache/airflow/blob/2.9.0/airflow/sensors/external_task.py#L352
WorkflowTrigger -> _get_count
https://github.com/apache/airflow/blob/2.9.0/airflow/triggers/external_task.py#L125
_get_count queries for execution_date
https://github.com/apache/airflow/blob/2.9.0/airflow/utils/sensor_helper.py#L93
Would it be possible to have a floor the datetime on day or hour
The use case I have is when a manual run is done (rather than re-running failed run) the lookup will fail. Since I know the runs are daily, as long as there has been a successful run on that day it would be good enough.
Beta Was this translation helpful? Give feedback.
All reactions