Skip to content

Idempotency: downstream rollup behavior when an upstream partition is cleared and re-run #65923

@Lee-W

Description

@Lee-W

Body

What

When a downstream rollup has already fired for a window that included an upstream partition, and that upstream partition is later cleared and re-run, the framework needs a defined behavior. Example: the monthly rollup for 2026-03 already fired with 2026-03-15 included; user clears and re-runs 2026-03-15. Does the rollup reprocess, hold, or no-op?

Open question

Auto re-trigger the rollup, require explicit user action, or expose a per-timetable policy?

Related

#59294

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:metaHigh-level information important to the community

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions