Introduce DatasetOrTimeSchedule
#36710
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sunank200
force-pushed
the
dataset-timetable
branch
from
7d5c320
to
f732cb4
Compare
sunank200
force-pushed
the
dataset-timetable
branch
from
4719029
to
2202ccf
Compare
uranusjr
requested review from
potiuk,
kaxil,
ashb,
bolkedebruin and
XD-DENG
as code owners
sunank200
force-pushed
the
dataset-timetable
branch
from
062bd5b
to
c8c8f1c
Compare
sunank200
requested review from
phanikumv,
utkarsharma2,
pankajkoti,
pankajastro and
dstandish
sunank200
force-pushed
the
dataset-timetable
branch
from
c8c8f1c
to
422fca8
Compare
sunank200
force-pushed
the
dataset-timetable
branch
from
422fca8
to
053133b
Compare
dstandish
reviewed
Jan 24, 2024
pankajkoti
reviewed
Jan 25, 2024
sunank200
force-pushed
the
dataset-timetable
branch
from
053133b
to
d9ba54e
Compare
phanikumv
changed the title
DatasetOrTimeSchedule
dstandish
reviewed
Jan 30, 2024
dstandish
approved these changes
Jan 30, 2024
potiuk
reviewed
Jan 30, 2024
potiuk
approved these changes
Jan 30, 2024
|
@sunank200 perhaps (either in this PR or followup) we should update the UI so that on home page it should shew the next run time instead of 
|
sunank200
force-pushed
the
dataset-timetable
branch
2 times, most recently
from
9f22bbe
to
b63cd00
Compare
potiuk
approved these changes
Jan 31, 2024
This special timetable allows a DAG to be run against a time-based schedule and dataset events at the same time. The logic is nothing special---scheduled runs are created based on a time-based timetable, and dataset-triggered runs are created when dataset events happen. The two do not interact in any way.
sunank200
force-pushed
the
dataset-timetable
branch
from
b63cd00
to
c9e9d0b
Compare
phanikumv
approved these changes
Feb 1, 2024
abhishekbhakat
pushed a commit
to abhishekbhakat/my_airflow
that referenced
this pull request
Mar 5, 2024
* Introduce DatasetTimetable This special timetable allows a DAG to be run against a time-based schedule and dataset events at the same time. The logic is nothing special---scheduled runs are created based on a time-based timetable, and dataset-triggered runs are created when dataset events happen. The two do not interact in any way. Co-authored-by: Ankit Chaurasia <8670962+sunank200@users.noreply.github.com> Co-authored-by: Daniel Standish <15932138+dstandish@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This special timetable allows a DAG to be run against a time-based schedule and dataset events at the same time. The logic is nothing special---scheduled runs are created based on a time-based timetable, and dataset-triggered runs are created when dataset events happen. The two do not interact in any way.
I aim to maybe refactor this a bit so a DAG can take more complex combinations (e.g.
dataset1 OR dataset2 OR timetable), but this is the simplest thing for the most common use case of “rerun DAG whenever things update, but also periodically once in a while”.Please suggest a better name for this class.