-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[docs] - Add guide for schedule timezone execution (DOC-238) (#22034)
## Summary & Motivation This PR splits the timezone content in the Schedules concept page out into its own guide. ## How I Tested These Changes eyes, local, `dagster dev` for examples
- Loading branch information
1 parent
37d139a
commit b596be9
Showing
5 changed files
with
158 additions
and
3 deletions.
There are no files selected for viewing
150 changes: 150 additions & 0 deletions
150
docs/content/concepts/automation/schedules/customizing-executing-timezones.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,150 @@ | ||
--- | ||
title: "Customizing a schedule's executing timezone | Dagster Docs" | ||
description: "By default, schedules without a set timezone execute in UTC. Learn to specify custom timezones on your Dagster schedules." | ||
--- | ||
|
||
# Customizing a schedule's executing timezone | ||
|
||
[Schedules](/concepts/partitions-schedules-sensors/schedules) that don't have a set timezone will, by default, execute in [UTC](https://en.wikipedia.org/wiki/Coordinated_Universal_Time). By the end of this guide, you'll know how to: | ||
|
||
- Set custom timezones on schedule definitions | ||
- Set custom timezones on partitioned jobs | ||
- Account for the impact of Daylight Savings Time on schedule execution times | ||
|
||
--- | ||
|
||
## Prerequisites | ||
|
||
To follow this guide, you need to be familiar with: | ||
|
||
- [Schedules](/concepts/partitions-schedules-sensors/schedules) | ||
- Jobs, either [asset](/concepts/assets/asset-jobs) or [op-based](/concepts/ops-jobs-graphs/op-jobs) | ||
- [Partitions](/concepts/partitions-schedules-sensors/partitions) (Optional) | ||
|
||
--- | ||
|
||
## Setting timezones on schedule definitions | ||
|
||
Using the `execution_timezone` parameter allows you to specify a timezone for the schedule on the following objects: | ||
|
||
- <PyObject object="schedule" decorator /> | ||
- <PyObject object="ScheduleDefinition" /> | ||
- <PyObject object="build_schedule_from_dbt_selection" module="dagster_dbt" /> | ||
|
||
This parameter accepts any [`tz` timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). For example, the following schedule will execute **every day at 9:00 AM in US Pacific time (America/Los_Angeles)**: | ||
|
||
```python file=concepts/partitions_schedules_sensors/schedules/schedules.py startafter=start_timezone endbefore=end_timezone | ||
my_timezone_schedule = ScheduleDefinition( | ||
job=my_job, cron_schedule="0 9 * * *", execution_timezone="America/Los_Angeles" | ||
) | ||
``` | ||
|
||
--- | ||
|
||
## Setting timezones on partitioned jobs | ||
|
||
Schedules constructed from partitioned jobs execute in the timezone defined on the partition's config. Partitions definitions have a `timezone` parameter, which accepts any [`tz` timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). | ||
|
||
For example, the following partition uses the **US Pacific (America/Los_Angeles)** timezone: | ||
|
||
```python file=concepts/partitions_schedules_sensors/partition_with_timezone.py | ||
from dagster import DailyPartitionsDefinition | ||
|
||
daily_partition = DailyPartitionsDefinition( | ||
start_date="2024-05-20", timezone="America/Los_Angeles" | ||
) | ||
``` | ||
|
||
--- | ||
|
||
## Execution times and Daylight Savings Time | ||
|
||
When Daylight Savings Time (DST) begins and ends, there may be some impact on your schedules' execution times. | ||
|
||
### Impact on daily schedules | ||
|
||
Because of DST transitions, it's possible to specify an execution time that doesn't exist for every scheduled interval. | ||
|
||
Let's say you have a **schedule that executes every day at 2:30 AM.** On the day DST begins, time jumps from 2:00AM to 3:00AM, which means the time of 2:30 AM won't exist. | ||
|
||
Dagster would instead run the schedule at the next time that exists, which would be 3:00 AM: | ||
|
||
```markdown | ||
# DST begins: time jumps forward an hour at 2:00 AM | ||
|
||
- 12:30 AM | ||
- 1:00 AM | ||
- 1:30 AM | ||
- 3:00 AM ## time transition; schedule executes | ||
- 3:30 AM | ||
- 4:00 AM | ||
``` | ||
|
||
It's also possible to specify an execution time that exists twice on one day every year. | ||
|
||
Let's say you have a **schedule that executes every day at 1:30 AM.** On the day DST ends, the hour from 1:00 AM to 2:00 AM repeats, which means the time of 1:30 AM will exist twice. This means there are two possible times the schedule could run. | ||
|
||
In this case, Dagster would execute the schedule at the second iteration of 1:30 AM: | ||
|
||
```markdown | ||
# DST ends: time jumps backward an hour at 2:00 AM | ||
|
||
- 12:30 AM | ||
- 1:00 AM | ||
- 1:30 AM | ||
- 1:00 AM ## time transition | ||
- 1:30 AM ## schedule executes | ||
- 2:00 AM | ||
``` | ||
|
||
### Impact on hourly schedules | ||
|
||
Hourly schedules are unaffected by daylight savings time transitions. Schedules will continue to run exactly once an hour, even as DST ends and the hour from 1:00 AM to 2:00 AM repeats. | ||
|
||
Let's say you have a **schedule that executes hourly at 30 minutes past the hour.** On the day DST ends, the schedule would run at 12:30 AM and both instances of 1:30 AM before proceeding normally at 2:30 AM: | ||
|
||
```markdown | ||
# DST ends: time jumps backward an hour at 2:00 AM | ||
|
||
- 12:30 AM ## schedule executes | ||
- 1:00 AM | ||
- 1:30 AM ## schedule executes | ||
- 1:00 AM ## time transition | ||
- 1:30 AM ## schedule executes | ||
- 2:00 AM | ||
- 2:30 AM ## schedule executes | ||
``` | ||
|
||
--- | ||
|
||
## APIs in this guide | ||
|
||
| Name | Description | | ||
| ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | ||
| <PyObject object="schedule" decorator /> | Decorator that defines a schedule that executes according to a given cron schedule. | | ||
| <PyObject object="ScheduleDefinition" /> | Class for schedules. | | ||
| <PyObject object="build_schedule_from_partitioned_job" /> | A function that constructs a schedule whose interval matches the partitioning of a partitioned job. | | ||
| <PyObject object="build_schedule_from_dbt_selection" module="dagster_dbt" /> | A function that constructs a schedule that materializes a set of specified dbt resources. | | ||
|
||
--- | ||
|
||
## Related | ||
|
||
<ArticleList> | ||
<ArticleListItem | ||
title="Troubleshooting schedules" | ||
href="/concepts/automation/schedules/troubleshooting" | ||
></ArticleListItem> | ||
<ArticleListItem | ||
title="Automating assets using schedules and jobs" | ||
href="/concepts/automation/schedules/automating-assets-schedules-jobs" | ||
></ArticleListItem> | ||
<ArticleListItem | ||
title="Automating ops using schedules and jobs" | ||
href="/concepts/automation/schedules/automating-ops-schedules-jobs" | ||
></ArticleListItem> | ||
<ArticleListItem | ||
title="Partitions" | ||
href="/concepts/partitions-schedules-sensors/partitions" | ||
></ArticleListItem> | ||
</ArticleList> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 changes: 5 additions & 0 deletions
5
...s_snippets/docs_snippets/concepts/partitions_schedules_sensors/partition_with_timezone.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
from dagster import DailyPartitionsDefinition | ||
|
||
daily_partition = DailyPartitionsDefinition( | ||
start_date="2024-05-20", timezone="America/Los_Angeles" | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
b596be9
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deploy preview for dagster-docs ready!
✅ Preview
https://dagster-docs-9aq6nlx8p-elementl.vercel.app
https://master.dagster.dagster-docs.io
Built with commit b596be9.
This pull request is being automatically deployed with vercel-action