[DISCUSS] Allow manual/operator-triggered DAG runs with future logical_date

[ZhaoMJ]

Hi all,

I'd like to discuss a change proposed in PR #65856 that relaxes the blanket ban on future logical_date for manually triggered and operator-triggered DAG runs.

Background

PR #46663 removed the allow_trigger_in_future config and hardcoded a block on any DAG run whose logical_date is in the future. This affects both the scheduler (_schedule_dag_run) and the RunnableExecDateDep task dependency. The result: when a user triggers a DAG with a future logical_date, the DagRun appears as "running" but tasks do not execute until the logical_date is reached.

This is blocking our migration from Airflow 2 to 3 — we use logical_date to represent business session dates that don't align with calendar dates, and these runs need to execute immediately at trigger time.

Given that run_after is used for scheduling and logical_date is meant to "not contain any semantics, but is simply a value for logical identification," I believe it makes sense to relax the block on future logical_date. Please see the PR description for the rationale, use cases, and more details.

Proposed change

Skip the future logical_date block for MANUAL and OPERATOR_TRIGGERED run types only. Scheduled runs remain blocked. The existing run_after <= now() query filter still controls when the scheduler picks up a run, but dag runs with future logical_date will be able to execute immediately if run_after is set to None or now(). A "Run immediately" checkbox in the Trigger DAG UI gives users explicit control. Also added run_after to TriggerDagRunOperator and TriggerDAGRunPayload so that users can control this from the operator path as well.

Why I believe this is non-breaking

The default behavior is unchanged. The run_after field already controls when the scheduler picks up a run. Manual triggers via the Core API already allow setting run_after=now() with a future logical_date — this just removes the secondary block that prevents it from working.

Broader question raised in review

logical_date is still used for TI prioritization in _executable_task_instances_to_queued and as a scheduling guard in _schedule_dag_run. Given that logical_date is defined as purely identificational ("does not contain any semantics") while run_after is defined as the scheduling control, should we decouple logical_date from scheduling entirely? I think that's a separate, larger discussion — this PR intentionally avoids changing TI ordering.

Looking for feedback on whether the approach is reasonable and if the broader decoupling warrants its own discussion thread.

Thanks,
Mingjie Zhao