Skip to content

Fix scheduler crash on log_filename_template with null logical_date#68149

Open
Vamsi-klu wants to merge 3 commits into
apache:mainfrom
Vamsi-klu:fix/log-filename-none-logical-date-68075
Open

Fix scheduler crash on log_filename_template with null logical_date#68149
Vamsi-klu wants to merge 3 commits into
apache:mainfrom
Vamsi-klu:fix/log-filename-none-logical-date-68075

Conversation

@Vamsi-klu
Copy link
Copy Markdown
Contributor

@Vamsi-klu Vamsi-klu commented Jun 7, 2026

What

Make the scheduler-side log filename renderer (log_filename_template_renderer in airflow-core/src/airflow/utils/helpers.py) tolerate a null logical_date:

  • Derive the date as ti.logical_date or ti.run_after (run_after is non-nullable on DagRun), so the renderer never dereferences None.
  • Expose ts and ts_nodash to Jinja templates, so templates can reference a stable timestamp without touching ti.logical_date directly.
  • Wrap rendering so that if a user-supplied template still raises, the renderer logs a warning and falls back to the default run_id-based template instead of propagating the exception.

Why

Since AIP-83, DagRun.logical_date (and therefore ti.logical_date) is None for asset-triggered / partitioned runs (#68075). When a custom log_filename_template references ti.logical_date — e.g. the reported ts_nodash | default(ti.logical_date.strftime("%Y%m%dT%H%M%S")) — rendering raises 'None' has no attribute 'strftime'. This happens inside _enqueue_task_instances_with_queued_state, in the scheduler critical section, so the whole scheduler crashes rather than just one task failing. Jinja's default() evaluates its argument eagerly, so exposing variables alone cannot rescue a template that calls ti.logical_date.strftime(...) directly — hence the additional fallback path.

The ti.logical_date or ti.run_after derivation matches the read side, FileTaskHandler._render_filename (airflow-core/src/airflow/utils/log/file_task_handler.py, which uses dag_run.logical_date or dag_run.run_after), so the write and read paths resolve to the same log path.

Impact

  • The scheduler no longer crashes when building a task's log filename for runs without a logical_date.
  • Existing well-formed templates are unaffected; for null-logical_date runs, logical_date/ts/ts_nodash now resolve to run_after instead of erroring.
  • A template that cannot render at all degrades to the default run_id-based filename (with a logged warning) rather than aborting scheduling — log paths for such tasks may differ from the custom template until the template is fixed.
  • Guidance (also captured in the newsfragment): prefer run_id or ts over ti.logical_date in log_filename_template.
  • Adds airflow-core/newsfragments/68149.bugfix.rst documenting the user-facing fix.

Testing

Adds TestLogFilenameTemplateRenderer in airflow-core/tests/unit/utils/test_helpers.py (db tests; the template is injected via the AIRFLOW__LOGGING__LOG_FILENAME_TEMPLATE env var because the default template contains %, which configparser interpolation cannot round-trip):

  • test_jinja_ts_is_none_safe (parametrized over logical_date set and None): a {{ ts }} template renders to the run_after-derived timestamp without crashing.
  • test_fstring_logical_date_none_uses_run_after: an f-string {logical_date} template with logical_date=None falls back to run_after.
  • test_broken_template_does_not_crash_scheduler: the exact failing template from ti.logical_date is not populated on scheduler ti logging #68075 (calling ti.logical_date.strftime(...) via default()) with logical_date=None must not raise; the renderer falls back to the default and the result contains the run_id.
  • test_write_path_matches_read_path (parametrized over Jinja and f-string templates, logical_date set and None): asserts the write-side renderer and FileTaskHandler._render_filename produce identical paths.

closes: #68075


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.8)

Generated-by: Claude Code (Opus 4.8) following the guidelines

Vamsi-klu pushed a commit to Vamsi-klu/airflow that referenced this pull request Jun 7, 2026
The check-newsfragment-pr-number CI check requires the newsfragment file
to be named after the PR number, not the issue number.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Vamsi-klu
Copy link
Copy Markdown
Contributor Author

This fixes a scheduler crash when rendering log_filename_template for runs with a None logical_date (AIP-83 asset-triggered / partitioned runs). The renderer now derives the date as ti.logical_date or ti.run_after — mirroring FileTaskHandler._render_filename on the read side — and degrades to the default run_id template instead of aborting the scheduling loop.

cc @uranusjr @jscheffl — would appreciate a review when you have time (no code owner is registered for utils/helpers.py, so tagging the most relevant recent maintainers).

The check-newsfragment-pr-number CI check requires the newsfragment file
to be named after the PR number, not the issue number.
@Vamsi-klu Vamsi-klu force-pushed the fix/log-filename-none-logical-date-68075 branch from 65f69a7 to ce887a5 Compare June 7, 2026 01:05
The check-newsfragments-are-valid hook requires non-significant
newsfragments to be a single line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ti.logical_date is not populated on scheduler ti logging

1 participant