Skip to content

Add target_date — a user-defined processing date for Dag runs#67329

Draft
dsuhinin wants to merge 4 commits into
apache:mainfrom
dsuhinin:dsuhinin/support-target-date
Draft

Add target_date — a user-defined processing date for Dag runs#67329
dsuhinin wants to merge 4 commits into
apache:mainfrom
dsuhinin:dsuhinin/support-target-date

Conversation

@dsuhinin
Copy link
Copy Markdown
Contributor

Motivation

Airflow's logical_date was designed as a scheduling identifier, not as a semantic "what date are we processing?" signal. In practice, teams often need to tell a task "process data for 2025-03-31" independently of when the run
was scheduled. The common workaround — storing a date in dag_run.conf — has no first-class template variable, no API surface, and no schema validation. target_date fills this gap.


What this PR adds

Dag-level target_date

Set a fixed date or a callable on the DAG object:

from datetime import date, timedelta
from airflow.sdk import DAG

# Fixed date — every run targets 2025-03-31
with DAG("my_dag", schedule="@daily", target_date=date(2025, 3, 31)) as dag:
    ...

# Callable — last day of the previous month
def last_day_of_prev_month(logical_date, data_interval_start, data_interval_end):
    return (logical_date.replace(day=1) - timedelta(days=1)).date()

with DAG("monthly_dag", schedule="@monthly", target_date=last_day_of_prev_month) as dag:
    ...

Task-level override

Individual tasks can refine the Dag-level value via their own callable.
The callable receives the full template context, so ctx["target_date"]
is already the Dag-level result:

def shift_one_week(ctx):
    return ctx["target_date"] - timedelta(weeks=1)

with DAG("my_dag", schedule="@weekly", target_date=last_day_of_prev_month) as dag:
    PythonOperator(
        task_id="weekly_lookback",
        python_callable=process,
        target_date=shift_one_week,
    )

Template variable {{ target_date }}

Resolution order

Nothing set anywherelogical_date.date()
DAG-level set onlydag-level value (snapshotted on the DagRun at creation time)
Task-level callable settask_callable(ctx)  where ctx["target_date"] is the dag-level value

REST API

target_date is a first-class field on all DagRun endpoints:

# Trigger with an explicit processing date
curl -X POST ".../dags/my_dag/dagRuns" \
     -d '{"target_date": "2025-03-31"}'

# Update after the fact
curl -X PATCH ".../dags/my_dag/dagRuns/my_run_id" \
     -d '{"target_date": "2025-04-30"}'

UI

The Dag Run Details page shows a read-only Target date row when the value is set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant