Skip to content

Support timezone in SDK temporal partition mappers#67164

Open
Lee-W wants to merge 2 commits into
apache:mainfrom
astronomer:fix-sdk-temporal-mapper-timezone
Open

Support timezone in SDK temporal partition mappers#67164
Lee-W wants to merge 2 commits into
apache:mainfrom
astronomer:fix-sdk-temporal-mapper-timezone

Conversation

@Lee-W
Copy link
Copy Markdown
Member

@Lee-W Lee-W commented May 19, 2026

Why

The Core _BaseTemporalMapper accepts a timezone kwarg, but the SDK class Dag authors instantiate (from airflow.sdk import StartOfDayMapper) did not — and the _Serializer dispatch dropped _timezone even if it had. So StartOfDayMapper(timezone="Asia/Taipei") silently fell back to UTC after serialization.

What

  • Add a keyword-only timezone kwarg to the SDK _BaseTemporalMapper.__init__, matching the Core signature.
  • Normalise string timezone values via parse_timezone() at construction so unknown names raise pendulum.tz.exceptions.InvalidTimezone eagerly, and _timezone matches Core's type contract (Timezone | FixedTimezone) instead of staying as a str.
  • Carry _timezone through the _Serializer dispatch handler for all six StartOf*Mapper classes so the Core class receives it on deserialize.
  • Unify the StartOfHourMapper import in encoders.py so the singledispatch registration matches the runtime type used by Dag authors.
  • Add a significant newsfragment documenting the keyword-only constructor as a breaking change against task-sdk/1.2.1.

Breaking changes

The SDK _BaseTemporalMapper constructor is now keyword-only. Callers that relied on StartOfDayMapper("%Y-%m-%dT%H:%M:%S") (valid in task-sdk 1.2.1) must switch to StartOfDayMapper(input_format="%Y-%m-%dT%H:%M:%S"). See airflow-core/newsfragments/67164.significant.rst.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: [Claude] following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@Lee-W Lee-W moved this to In Progress in AIP-76 Asset Partitioning May 19, 2026
@Lee-W Lee-W force-pushed the fix-sdk-temporal-mapper-timezone branch from ca0c304 to 2c4fbf3 Compare May 19, 2026 13:29
@Lee-W Lee-W changed the title feat(partition-mapper): support timezone in SDK temporal mappers feat: support timezone in SDK temporal partition mappers May 19, 2026
@Lee-W Lee-W marked this pull request as ready for review May 19, 2026 13:32
@Lee-W Lee-W moved this from In Progress to In Review in AIP-76 Asset Partitioning May 19, 2026
@Lee-W Lee-W force-pushed the fix-sdk-temporal-mapper-timezone branch from 2c4fbf3 to 408f7a0 Compare May 19, 2026 13:39

def __init__(
self,
*,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding *, here makes the SDK constructor keyword-only. That matches the Core class shape, which is the right destination, but it's a breaking signature change against the already-released task-sdk/1.2.1, where _BaseTemporalMapper.__init__ accepts input_format positionally — so StartOfDayMapper("%Y-%m-%dT%H:%M:%S") is valid public API today and would break on upgrade. Worth calling out in the PR description (and a newsfragment) since this is the public entry point Dag authors import via from airflow.sdk import StartOfDayMapper.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just added it newsfragment and description. Should we include it in 3.3.0 or 3.2.3? interface inconsistency can probably(?) be considered as a bug.

Comment thread task-sdk/src/airflow/sdk/definitions/partition_mappers/temporal.py Outdated
Copy link
Copy Markdown
Member

@uranusjr uranusjr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit on timezone input handling in a thread above

Lee-W added 2 commits May 20, 2026 17:03
Add a keyword-only `timezone` kwarg to the SDK `_BaseTemporalMapper`, carry `_timezone` through `_Serializer` dispatch, and resolve string values via `parse_timezone()` at construction. Breaking change vs `task-sdk/1.2.1`; see `67164.significant.rst`.
@Lee-W Lee-W force-pushed the fix-sdk-temporal-mapper-timezone branch from 408f7a0 to 8da1ce1 Compare May 20, 2026 09:04
@Lee-W Lee-W changed the title feat: support timezone in SDK temporal partition mappers Support timezone in SDK temporal partition mappers May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: In Review

Development

Successfully merging this pull request may close these issues.

3 participants