Fix EcsRunTaskOperator deferred logs read from the wrong region - #70474
Merged
Conversation
boring-cyborg
Bot
added
area:providers
provider:amazon
2 tasks
SameerMesiah97
left a comment
SameerMesiah97
left a comment
Contributor
There was a problem hiding this comment.
Looks good to me. I just have one comment but its minor.
TaskDoneTrigger received a single region_name and used it to build both the ECS waiter client and the CloudWatch client, so a task definition shipping its logs to a different region got no logs forwarded while the task was deferred -- only repeated "didn't exist (yet)" messages for the whole run. The non-deferrable path and the post-deferral fetch already resolve awslogs_region; this brings the in-deferral forwarding in line with them. closes: apache#70465
ColtenOuO
force-pushed
the
fix-ecs-trigger-awslogs-region
branch
from
July 27, 2026 01:08
0aa13df to
92ff382
Compare
shahar1
approved these changes
Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When
EcsRunTaskOperatorruns withdeferrable=Trueand the task definition ships its container logs to a different region than the one the task runs in (awslogs_region!=region_name), no logs are forwarded during deferral -- the trigger just repeats "Tried to get logs from stream ... but it didn't exist (yet)" for the whole task duration.TaskDoneTriggerwas given a singleregion_nameand used it for two different clients: the ECS waiter (correct) and the CloudWatchget_log_eventsclient (wrong -- the log group lives in the other region).The operator never passed the logs region to the trigger, so the trigger had no way to tell them apart.
This adds a
log_region_nameparameter toTaskDoneTrigger, used only for theAwsLogsHook, and has the operator passresolve_awslogs_region()when deferring. TheEcsHookkeeps usingregion_name.The parameter defaults to
Noneand falls back toregion_name, so triggers that were already serialized before this change deserialize without the key and keep their current behaviour.This is the remaining half of the log-region resolution: #70464 fixed the post-deferral fetch in
execute_complete, and the non-deferrable path already usedresolve_awslogs_region()via_get_task_log_fetcher().closes: #70465
Was generative AI tooling used to co-author this PR?