Add is_backfillable property to DAG API responses#64644
Add is_backfillable property to DAG API responses#64644Dev-iL wants to merge 1 commit intoapache:mainfrom
is_backfillable property to DAG API responses#64644Conversation
There was a problem hiding this comment.
Pull request overview
This PR improves backfill UX by exposing whether a DAG’s schedule supports backfilling via a new is_backfillable field in DAG API responses, enforcing non-periodic schedule rejection in backfill endpoints, and updating the UI to disable backfill when unsupported.
Changes:
- Add computed
is_backfillableto DAG-related API response models and OpenAPI specs (public + UI). - Validate backfills against
dag.timetable.periodic(rejectingNone,@once,@continuous, asset-triggered, partitioned asset schedules) and rename the related exception. - Update Trigger DAG modal logic and i18n to use
is_backfillable, plus add regression/unit tests.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
uv.lock |
Updates lockfile metadata/deps (includes OAuth/authlib-related changes). |
airflow-ctl/src/airflowctl/api/datamodels/generated.py |
Adds is_backfillable to generated CLI client DAG response models. |
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py |
Introduces computed is_backfillable on DAGResponse (and inheritors). |
airflow-core/src/airflow/models/backfill.py |
Renames schedule exception + switches backfill validation to timetable.periodic. |
airflow-core/src/airflow/api_fastapi/core_api/routes/public/backfills.py |
Updates route exception handling to the renamed exception. |
airflow-core/src/airflow/api_fastapi/core_api/openapi/v2-rest-api-generated.yaml |
Publishes is_backfillable in public OpenAPI schema for DAG responses. |
airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml |
Publishes is_backfillable in private UI OpenAPI schema for DAG responses. |
airflow-core/src/airflow/ui/openapi-gen/requests/types.gen.ts |
Updates generated TS types to include is_backfillable. |
airflow-core/src/airflow/ui/openapi-gen/requests/schemas.gen.ts |
Updates generated TS schemas to include is_backfillable as required/readOnly. |
airflow-core/src/airflow/ui/src/components/TriggerDag/TriggerDAGModal.tsx |
Disables/gates Backfill option using dag.is_backfillable instead of hasSchedule. |
airflow-core/src/airflow/ui/public/i18n/locales/en/components.json |
Replaces tooltip string with scheduleNotBackfillable message. |
airflow-core/tests/unit/models/test_backfill.py |
Adds coverage for rejecting non-periodic schedules in create/dry-run helpers. |
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_backfills.py |
Updates validation expectations for non-periodic schedules. |
airflow-core/tests/unit/api_fastapi/core_api/datamodels/test_dags.py |
Adds unit tests for DAGResponse.is_backfillable computation. |
airflow-core/tests/unit/api_fastapi/core_api/datamodels/__init__.py |
Adds package init for new datamodel tests directory. |
b45539e to
33e3f8f
Compare
pierrejeambrun
left a comment
There was a problem hiding this comment.
A few suggestions, otherwise looking good to me
| _NON_BACKFILLABLE_SUMMARIES: frozenset[str | None] = frozenset( | ||
| {None, "@once", "@continuous", "Asset", "Partitioned Asset"} | ||
| ) |
There was a problem hiding this comment.
This will diverge from the .periodic attribute (what access here in the API i.e timetable_summary vs the timetable.periodic). Is it possible similarly to the timetable_partitioned to store that in the DagModel via a migration ?
There was a problem hiding this comment.
It's still better than the current null table matching only. But will hold problems for custom timetables.
There was a problem hiding this comment.
Done. The string-matching approach (_NON_BACKFILLABLE_SUMMARIES frozenset) has been replaced entirely:
- Migration 0111 adds a
timetable_periodicBoolean column to thedagtable (same pattern astimetable_partitioned—server_default="0",nullable=False). collection.pysetsdm.timetable_periodic = dag.timetable.periodicduring DAG sync, right next totimetable_partitioned.DAGResponse.is_backfillablenow reads fromself.timetable_periodic(the DB column) instead of string-matchingtimetable_summary. This means custom timetables that setperiodic = Falseare handled correctly.- The backend checks in
_do_dry_runand_create_backfillcontinue to usedag.timetable.periodicfrom the serialized DAG directly — both derive from the same source.
| ); | ||
|
|
||
| const isBackfillable = dag?.is_backfillable ?? false; | ||
| const hasSchedule = dag?.timetable_summary !== null; |
There was a problem hiding this comment.
Is hasSchedule still needed? Or should we use isBackfillable now?
There was a problem hiding this comment.
hasSchedule is still needed as it serves a different purpose. It's passed to TriggerDAGForm (line 144) where it controls whether the data interval date pickers are shown in single-run trigger mode. A Dag with @once or @continuous schedule still has a schedule (so hasSchedule = true, data interval pickers shown), but isn't backfillable (isBackfillable = false, backfill radio card disabled).
In short: hasSchedule = "does this DAG have any schedule at all?" (controls data interval UI), isBackfillable = "can this DAG be backfilled?" (controls backfill radio card).
| def is_backfillable(self) -> bool: | ||
| """Whether this DAG's schedule supports backfilling.""" | ||
| return self.timetable_summary not in self._NON_BACKFILLABLE_SUMMARIES |
There was a problem hiding this comment.
Correct me if I am wrong but I think this can be confusing given also allowed_run_types which was added in #61833
There was a problem hiding this comment.
Addressed as follows: is_backfillable now unifies both concerns into a single source of truth for the UI:
@computed_field
@property
def is_backfillable(self) -> bool:
if not self.timetable_periodic:
return False
if self.allowed_run_types is not None and DagRunType.BACKFILL_JOB not in self.allowed_run_types:
return False
return TrueSo is_backfillable is False when:
- The schedule is non-periodic (asset-triggered,
@once,@continuous, no-schedule), OR allowed_run_typesexplicitly excludesBACKFILL_JOB
The UI only needs to check is_backfillable (it doesn't need to reason about both timetable_periodic and allowed_run_types separately). The backend also checks both: _do_dry_run now validates allowed_run_types (it previously only checked this in _create_backfill), so dry-run and create are consistent.
33e3f8f to
398b05d
Compare
2c8c69a to
e733faa
Compare
e733faa to
d888c11
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 44 out of 44 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (2)
airflow-core/tests/unit/api_fastapi/core_api/datamodels/test_dags.py:1
DAGResponse.ownersis typed aslist[str](and the OpenAPI/TS types reflect an array). Providing a bare string risks validation failure or unintended coercion (e.g., into a list of characters), making these tests flaky/incorrect. Change the default to a list such as["airflow"].
airflow-core/src/airflow/ui/public/i18n/locales/zh-CN/components.json:1- Many non-English locale files introduce the new
scheduleNotBackfillablemessage in English, which is a localization regression compared to the removed translatedtooltip. Consider translating this new string per locale (or reusing the prior locale-specific tooltip phrasing adapted to the new meaning) so users don’t see English text in localized UIs.
| "is_backfillable": core_timetable.periodic | ||
| and (dag.allowed_run_types is None or "backfill" in dag.allowed_run_types), |
| except ( | ||
| InvalidReprocessBehavior, | ||
| InvalidBackfillDirection, | ||
| DagNoScheduleException, | ||
| DagNonPeriodicScheduleException, | ||
| InvalidBackfillDate, | ||
| ) as e: |
| Raised when attempting to create backfill for a Dag with no schedule. | ||
| Raised when attempting to backfill a Dag whose schedule is fundamentally incompatible with backfills. | ||
|
|
||
| This covers the following timetables types: |
| "permissionDenied": "Dry Run Failed: User does not have permission to create backfills.", | ||
| "reprocessBehavior": "Reprocess Behavior", | ||
| "run": "Run Backfill", | ||
| "scheduleNotBackfillable": "This Dag's schedule does not support backfills", |
Context
Currently, when attempting to backfill a DAG that has an Asset schedule, after going to the backfill section in the trigger form and choosing dates we get an error saying: "No runs matching selected criteria." (on 2.11 it says "No run dates were found for the given dates and dag interval."). This is confusing UX-wise: instead of being shown right from the start (because it is tied to how the DAG is configured), it appears only after the user selects a date range. This sequence of events implies causality between the user's choice and the error — which is not true.
Additionally, DAGs that configure
allowed_run_typesto excludeBACKFILL_JOBhad no upfront indication that backfilling is disabled.Summary
timetable_periodicboolean column toDagModelvia Alembic migration (following thetimetable_partitionedpattern), set fromdag.timetable.periodicduring DAG sync.is_backfillablefield to DAG API responses that unifies both schedule compatibility (timetable_periodic) and run-type permissions (allowed_run_types) into a single source of truth.timetable_summary == "None"check with a propertimetable.periodiccheck in both_do_dry_runand_create_backfill, catching all non-periodic schedules (@once,@continuous, asset-triggered, partitioned asset) — not just unscheduled DAGs.allowed_run_typesvalidation to_do_dry_run(previously only in_create_backfill), ensuring dry-run and create return consistent errors.DagNoScheduleExceptiontoDagNonPeriodicScheduleExceptionto reflect the broader validation.is_backfillablefield instead of thehasScheduleheuristic, so the Backfill option is correctly disabled for all non-backfillable DAGs.Changes
Migration:
timetable_periodicBoolean column to thedagtable (server_default="0",nullable=False).dag_processing/collection.pysetsdm.timetable_periodic = dag.timetable.periodicduring DAG sync.API / Models:
DagModeldeclarestimetable_periodic: Mapped[bool].DAGResponse.is_backfillable— computed field:Trueonly whentimetable_periodic is TrueANDBACKFILL_JOBis permitted byallowed_run_types.backfill.py— both_create_backfilland_do_dry_runcheckdag.timetable.periodicandallowed_run_types.DagNoScheduleException->DagNonPeriodicScheduleException.dag_command.py—is_backfillablecomputed from bothperiodicandallowed_run_types.UI:
TriggerDAGModal.tsxusesis_backfillableto gate the Backfill radio option.hasScheduleis kept forTriggerDAGForm(controls data interval display — separate concern).backfill.tooltiptobackfill.scheduleNotBackfillablein all 21 locales).Tests:
TestIsBackfillabletests covering: non-periodic, periodic,allowed_run_types=None, backfill included/excluded, and the combined non-periodic+allowed case.test_create_backfill_non_periodic_schedule_rejectedandtest_do_dry_run_non_periodic_schedule_rejectedtests covering@once,@continuous,None, and asset schedules.test_no_schedule_dagfor new exception behavior.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Opus 4.6 following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.