Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Termination order of linked containers #474

Open
seiffert opened this Issue Aug 9, 2016 · 8 comments

Comments

Projects
None yet
@seiffert
Copy link
Contributor

seiffert commented Aug 9, 2016

We're experiencing an issue where linked containers in a task aren't terminated in the "correct" order. We would have expected that containers are terminated in a way that consumer containers are terminated before their linked containers.

In our case, a local redis cache is terminated before the consuming web application is. This in turn leads to exceptions in the web application.

Can you tell me whether this is expected behaviour or whether we can request the termination order to be changed? I've tried to dig into the agent's code a bit and think that I've found out that a task's containers aren't terminated in any particular order right now.

@kiranmeduri

This comment has been minimized.

Copy link
Contributor

kiranmeduri commented Aug 10, 2016

@seiffert thanks for pointing this out. You are right that termination order of containers within a task is non-deterministic. Note that it is deterministic when starting containers based on links and volumes (https://github.com/aws/amazon-ecs-agent/blob/master/agent/engine/dependencygraph/graph.go).

We will look into this and update the issue.

-kiran

@danvaida

This comment has been minimized.

Copy link

danvaida commented Feb 11, 2018

Any updates on this?

@blablacio

This comment has been minimized.

Copy link

blablacio commented Feb 20, 2018

@kiranmeduri Also interested in updates on the topic as some of our containers fail because their linked containers are stopped first.

@epiehl

This comment has been minimized.

Copy link

epiehl commented Nov 2, 2018

Still no updates? Our workflows are crashing as well due to caching containers being stopped before the applications gracefully exits.

@coultn

This comment has been minimized.

Copy link

coultn commented Nov 2, 2018

Thanks for checking in on this. I wanted to let you know that we on the ECS team are aware of this issue, and that it is under active consideration. +1's and additional details on use cases are always appreciated and will help inform our work moving forward.

@petderek

This comment has been minimized.

Copy link
Contributor

petderek commented Jan 18, 2019

We are looking at addressing this along with startup order in aws/containers-roadmap#123

@snowhork

This comment has been minimized.

Copy link

snowhork commented Jan 23, 2019

I opened PR #1809.

If this PR is acceptable, I will complete all testing and request review as soon as possible.

@abicky

This comment has been minimized.

Copy link

abicky commented Jan 31, 2019

We link our application container with a fluentd container. However the fluentd container always stops before the application container stops, so some data posted to fluentd lose.

I believe #1809 resolves the issue, so I appreciate if you take a look the pull request and post a comment, for example "This change looks good, so please complete all tests." or "We won't merge this pull request because it will conflicts with aws/containers-roadmap#123".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.