Skip to content

Add is_backfillable property to DAG API responses#64644

Open
Dev-iL wants to merge 1 commit intoapache:mainfrom
Dev-iL:2604/invalid_schedule
Open

Add is_backfillable property to DAG API responses#64644
Dev-iL wants to merge 1 commit intoapache:mainfrom
Dev-iL:2604/invalid_schedule

Conversation

@Dev-iL
Copy link
Copy Markdown
Collaborator

@Dev-iL Dev-iL commented Apr 2, 2026

Context

Currently, when attempting to backfill a DAG that has an Asset schedule, after going to the backfill section in the trigger form and choosing dates we get an error saying: "No runs matching selected criteria." (on 2.11 it says "No run dates were found for the given dates and dag interval."). This is confusing UX-wise: instead of being shown right from the start (because it is tied to how the DAG is configured), it appears only after the user selects a date range. This sequence of events implies causality between the user's choice and the error — which is not true.

Additionally, DAGs that configure allowed_run_types to exclude BACKFILL_JOB had no upfront indication that backfilling is disabled.

Summary

  • Adds a timetable_periodic boolean column to DagModel via Alembic migration (following the timetable_partitioned pattern), set from dag.timetable.periodic during DAG sync.
  • Adds a computed is_backfillable field to DAG API responses that unifies both schedule compatibility (timetable_periodic) and run-type permissions (allowed_run_types) into a single source of truth.
  • Replaces the backend's string-based timetable_summary == "None" check with a proper timetable.periodic check in both _do_dry_run and _create_backfill, catching all non-periodic schedules (@once, @continuous, asset-triggered, partitioned asset) — not just unscheduled DAGs.
  • Adds allowed_run_types validation to _do_dry_run (previously only in _create_backfill), ensuring dry-run and create return consistent errors.
  • Renames DagNoScheduleException to DagNonPeriodicScheduleException to reflect the broader validation.
  • Updates the UI to use the new is_backfillable field instead of the hasSchedule heuristic, so the Backfill option is correctly disabled for all non-backfillable DAGs.
image

Changes

Migration:

  • Migration 0111 adds timetable_periodic Boolean column to the dag table (server_default="0", nullable=False).
  • dag_processing/collection.py sets dm.timetable_periodic = dag.timetable.periodic during DAG sync.

API / Models:

  • DagModel declares timetable_periodic: Mapped[bool].
  • DAGResponse.is_backfillable — computed field: True only when timetable_periodic is True AND BACKFILL_JOB is permitted by allowed_run_types.
  • backfill.py — both _create_backfill and _do_dry_run check dag.timetable.periodic and allowed_run_types.
  • Renamed DagNoScheduleException -> DagNonPeriodicScheduleException.
  • dag_command.pyis_backfillable computed from both periodic and allowed_run_types.

UI:

  • TriggerDAGModal.tsx uses is_backfillable to gate the Backfill radio option. hasSchedule is kept for TriggerDAGForm (controls data interval display — separate concern).
  • Updated i18n strings (renamed backfill.tooltip to backfill.scheduleNotBackfillable in all 21 locales).

Tests:

  • New TestIsBackfillable tests covering: non-periodic, periodic, allowed_run_types=None, backfill included/excluded, and the combined non-periodic+allowed case.
  • New test_create_backfill_non_periodic_schedule_rejected and test_do_dry_run_non_periodic_schedule_rejected tests covering @once, @continuous, None, and asset schedules.
  • Updated existing test_no_schedule_dag for new exception behavior.
  • Updated test fixtures in DAG response tests, DagCard UI tests, and airflow-ctl tests.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Opus 4.6 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves backfill UX by exposing whether a DAG’s schedule supports backfilling via a new is_backfillable field in DAG API responses, enforcing non-periodic schedule rejection in backfill endpoints, and updating the UI to disable backfill when unsupported.

Changes:

  • Add computed is_backfillable to DAG-related API response models and OpenAPI specs (public + UI).
  • Validate backfills against dag.timetable.periodic (rejecting None, @once, @continuous, asset-triggered, partitioned asset schedules) and rename the related exception.
  • Update Trigger DAG modal logic and i18n to use is_backfillable, plus add regression/unit tests.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
uv.lock Updates lockfile metadata/deps (includes OAuth/authlib-related changes).
airflow-ctl/src/airflowctl/api/datamodels/generated.py Adds is_backfillable to generated CLI client DAG response models.
airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py Introduces computed is_backfillable on DAGResponse (and inheritors).
airflow-core/src/airflow/models/backfill.py Renames schedule exception + switches backfill validation to timetable.periodic.
airflow-core/src/airflow/api_fastapi/core_api/routes/public/backfills.py Updates route exception handling to the renamed exception.
airflow-core/src/airflow/api_fastapi/core_api/openapi/v2-rest-api-generated.yaml Publishes is_backfillable in public OpenAPI schema for DAG responses.
airflow-core/src/airflow/api_fastapi/core_api/openapi/_private_ui.yaml Publishes is_backfillable in private UI OpenAPI schema for DAG responses.
airflow-core/src/airflow/ui/openapi-gen/requests/types.gen.ts Updates generated TS types to include is_backfillable.
airflow-core/src/airflow/ui/openapi-gen/requests/schemas.gen.ts Updates generated TS schemas to include is_backfillable as required/readOnly.
airflow-core/src/airflow/ui/src/components/TriggerDag/TriggerDAGModal.tsx Disables/gates Backfill option using dag.is_backfillable instead of hasSchedule.
airflow-core/src/airflow/ui/public/i18n/locales/en/components.json Replaces tooltip string with scheduleNotBackfillable message.
airflow-core/tests/unit/models/test_backfill.py Adds coverage for rejecting non-periodic schedules in create/dry-run helpers.
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_backfills.py Updates validation expectations for non-periodic schedules.
airflow-core/tests/unit/api_fastapi/core_api/datamodels/test_dags.py Adds unit tests for DAGResponse.is_backfillable computation.
airflow-core/tests/unit/api_fastapi/core_api/datamodels/__init__.py Adds package init for new datamodel tests directory.

@Dev-iL Dev-iL force-pushed the 2604/invalid_schedule branch 7 times, most recently from b45539e to 33e3f8f Compare April 4, 2026 15:31
Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions, otherwise looking good to me

Comment on lines +131 to +133
_NON_BACKFILLABLE_SUMMARIES: frozenset[str | None] = frozenset(
{None, "@once", "@continuous", "Asset", "Partitioned Asset"}
)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will diverge from the .periodic attribute (what access here in the API i.e timetable_summary vs the timetable.periodic). Is it possible similarly to the timetable_partitioned to store that in the DagModel via a migration ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still better than the current null table matching only. But will hold problems for custom timetables.

Copy link
Copy Markdown
Collaborator Author

@Dev-iL Dev-iL Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The string-matching approach (_NON_BACKFILLABLE_SUMMARIES frozenset) has been replaced entirely:

  • Migration 0111 adds a timetable_periodic Boolean column to the dag table (same pattern as timetable_partitionedserver_default="0", nullable=False).
  • collection.py sets dm.timetable_periodic = dag.timetable.periodic during DAG sync, right next to timetable_partitioned.
  • DAGResponse.is_backfillable now reads from self.timetable_periodic (the DB column) instead of string-matching timetable_summary. This means custom timetables that set periodic = False are handled correctly.
  • The backend checks in _do_dry_run and _create_backfill continue to use dag.timetable.periodic from the serialized DAG directly — both derive from the same source.

);

const isBackfillable = dag?.is_backfillable ?? false;
const hasSchedule = dag?.timetable_summary !== null;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is hasSchedule still needed? Or should we use isBackfillable now?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hasSchedule is still needed as it serves a different purpose. It's passed to TriggerDAGForm (line 144) where it controls whether the data interval date pickers are shown in single-run trigger mode. A Dag with @once or @continuous schedule still has a schedule (so hasSchedule = true, data interval pickers shown), but isn't backfillable (isBackfillable = false, backfill radio card disabled).

In short: hasSchedule = "does this DAG have any schedule at all?" (controls data interval UI), isBackfillable = "can this DAG be backfilled?" (controls backfill radio card).

Comment on lines +138 to +140
def is_backfillable(self) -> bool:
"""Whether this DAG's schedule supports backfilling."""
return self.timetable_summary not in self._NON_BACKFILLABLE_SUMMARIES
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I am wrong but I think this can be confusing given also allowed_run_types which was added in #61833

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed as follows: is_backfillable now unifies both concerns into a single source of truth for the UI:

@computed_field
@property
def is_backfillable(self) -> bool:
    if not self.timetable_periodic:
        return False
    if self.allowed_run_types is not None and DagRunType.BACKFILL_JOB not in self.allowed_run_types:
        return False
    return True

So is_backfillable is False when:

  1. The schedule is non-periodic (asset-triggered, @once, @continuous, no-schedule), OR
  2. allowed_run_types explicitly excludes BACKFILL_JOB

The UI only needs to check is_backfillable (it doesn't need to reason about both timetable_periodic and allowed_run_types separately). The backend also checks both: _do_dry_run now validates allowed_run_types (it previously only checked this in _create_backfill), so dry-run and create are consistent.

@Dev-iL Dev-iL force-pushed the 2604/invalid_schedule branch from e733faa to d888c11 Compare April 9, 2026 21:32
@kaxil kaxil requested a review from Copilot April 10, 2026 19:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

airflow-core/tests/unit/api_fastapi/core_api/datamodels/test_dags.py:1

  • DAGResponse.owners is typed as list[str] (and the OpenAPI/TS types reflect an array). Providing a bare string risks validation failure or unintended coercion (e.g., into a list of characters), making these tests flaky/incorrect. Change the default to a list such as ["airflow"].
    airflow-core/src/airflow/ui/public/i18n/locales/zh-CN/components.json:1
  • Many non-English locale files introduce the new scheduleNotBackfillable message in English, which is a localization regression compared to the removed translated tooltip. Consider translating this new string per locale (or reusing the prior locale-specific tooltip phrasing adapted to the new meaning) so users don’t see English text in localized UIs.

Comment on lines +279 to +280
"is_backfillable": core_timetable.periodic
and (dag.allowed_run_types is None or "backfill" in dag.allowed_run_types),
Comment on lines 262 to 267
except (
InvalidReprocessBehavior,
InvalidBackfillDirection,
DagNoScheduleException,
DagNonPeriodicScheduleException,
InvalidBackfillDate,
) as e:
Raised when attempting to create backfill for a Dag with no schedule.
Raised when attempting to backfill a Dag whose schedule is fundamentally incompatible with backfills.

This covers the following timetables types:
"permissionDenied": "Dry Run Failed: User does not have permission to create backfills.",
"reprocessBehavior": "Reprocess Behavior",
"run": "Run Backfill",
"scheduleNotBackfillable": "This Dag's schedule does not support backfills",
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:airflow-ctl area:API Airflow's REST/HTTP API area:translations area:UI Related to UI/UX. For Frontend Developers. translation:default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants