Skip to content

Fix EcsRunTaskOperator deferred-logs region and template field init - #70464

Merged
shahar1 merged 1 commit into
apache:mainfrom
fat-catTW:fix-70296-ecs-run-task-template-init
Jul 26, 2026
Merged

Fix EcsRunTaskOperator deferred-logs region and template field init#70464
shahar1 merged 1 commit into
apache:mainfrom
fat-catTW:fix-70296-ecs-run-task-template-init

Conversation

@fat-catTW

@fat-catTW fat-catTW commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Move EcsRunTaskOperator template-field decisions out of __init__ so templated values are rendered before they affect ECS log handling.

This removes the operator from the validate_operators_init exemption list.

Besides moving the template-field logic out of __init__ (so templated awslogs_region/region_name values are rendered before they take effect), this also fixes a real bug in the deferrable path: execute_complete built its AwsLogsHook with region_name=self.region_name, so when awslogs_region was explicitly set to a different region than the task's region, the final log fetch after deferral queried CloudWatch in the wrong region and found no log stream. The new resolve_awslogs_region() helper is now the single place the log region is resolved (explicit awslogs_region, falling back to region_name), and both execute_complete and the task log fetcher use it. Note this covers only the post-deferral fetch — log forwarding during deferral still uses the task region inside TaskDoneTrigger; that remaining half is tracked in #70465.

related: #70296

Was generative AI tooling used to co-author this PR?
  • [X] Yes (please specify the tool below)

Generated-by: [Codex] following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Template fields are rendered after operator construction, so constructor-time reads can act on unresolved Jinja values. Keeping these decisions in the execution path lets templated values resolve before they affect ECS log handling.
@shahar1

shahar1 commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Nice! You might haven't notice, but you probably fixed another bug as well :)

@shahar1 shahar1 changed the title Fix EcsRunTaskOperator template field initialization Fix EcsRunTaskOperator deferred-logs region and template field init Jul 26, 2026
@shahar1
shahar1 merged commit 03f45e1 into apache:main Jul 26, 2026
83 checks passed
@shahar1 shahar1 removed the backport-to-v3-3-test Backport to v3-3-test label Jul 26, 2026
@fat-catTW

Copy link
Copy Markdown
Contributor Author

Thanks!
I missed that at first too, but yes, this also makes execute_complete() honor awslogs_region in deferrable mode instead of always using region_name when fetching the final CloudWatch log event.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants