Skip to content

Reject negative default_retention_days in task SDK and core API routes#67890

Merged
amoghrajesh merged 3 commits into
apache:mainfrom
astronomer:aip-103-min-validate-default_retention_days
Jun 3, 2026
Merged

Reject negative default_retention_days in task SDK and core API routes#67890
amoghrajesh merged 3 commits into
apache:mainfrom
astronomer:aip-103-min-validate-default_retention_days

Conversation

@amoghrajesh
Copy link
Copy Markdown
Contributor

@amoghrajesh amoghrajesh commented Jun 2, 2026


Was generative AI tooling used to co-author this PR?
  • Yes: claude sonnet 4.6

What?

[state_store] default_retention_days could be set to a negative integer without any error. The task sdk silently treated it as "never expire" (same as 0), and the API route would compute a negative timedelta, producing an expires_at in the past.

Current behaviour

A negative default_retention_days in config is accepted without complaint. In the sdk, days <= 0 mapped to expires_at=None — so -5 behaved identically to 0. In the API route, it produced an expires_at timestamp in the past, causing state rows to expire immediately.

Proposed change

  • SDK (context.py): set() raises ValueError with a clear message if default_retention_days < 0. days == 0 (disable expiry) continues to work.
  • API (task_state.py): _resolve_expires_at() returns HTTP 400 Bad Request if default_retention_days < 0. days == 0 correctly returns None (never expire).

Note: expires_at="default" with default_retention_days=0 now correctly stores expires_at=NULL (never expires) on the API path. Previously it computed now + timedelta(days=0) = now, causing rows to expire immediately, which contradicts both the sdk behaviour and the config docs ("Set to 0 to disable expiry").


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@amoghrajesh amoghrajesh changed the title Reject negative default_retention_days in task SDK and core API routes Reject negative default_retention_days in task SDK and core API routes Jun 2, 2026
Copy link
Copy Markdown
Collaborator

@jroachgolf84 jroachgolf84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question; are we now able to trigger a cleanup with the API? If so, I need to update my docs.

@amoghrajesh
Copy link
Copy Markdown
Contributor Author

Unrelated failures, merging

@amoghrajesh amoghrajesh merged commit 7a43d8b into apache:main Jun 3, 2026
170 of 176 checks passed
@amoghrajesh amoghrajesh deleted the aip-103-min-validate-default_retention_days branch June 3, 2026 04:37
@github-project-automation github-project-automation Bot moved this from In review to Done in AIP-103: Task State Management Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:task-sdk

Projects

Development

Successfully merging this pull request may close these issues.

4 participants