New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Depends_on in version3 #4305

Closed
nischay30 opened this Issue Jan 9, 2017 · 32 comments

Comments

Projects
None yet
@nischay30

nischay30 commented Jan 9, 2017

Hi i would like to know what is the alternate of depends_on in docker-compose v3 as in the release notes you said we will jot port the depends_on feature in v3

@shin-

This comment has been minimized.

Member

shin- commented Jan 9, 2017

depends_on still exists in v3, but healthcheck dependencies (and as a result, the extended syntax) will not be ported.

HTH

@shin- shin- added the kind/question label Jan 9, 2017

@nischay30

This comment has been minimized.

nischay30 commented Jan 10, 2017

But i have written a docker-compose v3 and i try to deploy on swarm but depends_on is not working as the container are not starting in the fashion in which they have to be started.

@shin-

This comment has been minimized.

Member

shin- commented Jan 10, 2017

Are you using docker-compose or docker stack deploy?

@nischay30

This comment has been minimized.

nischay30 commented Jan 11, 2017

I am using docker stack deploy
and i try to deploy it on swarm of 7 machines

@stffabi

This comment has been minimized.

stffabi commented Jan 18, 2017

What would be the equivalent on docker-compose v3 to achieve something like healthcheck dependencies? If you are going to drop that functionality in v3 one should basically never use it, or at least there should be a migration path for this.

What are the intentions to introduce it in a new docker-compose v2.1 and then dropping it for v3? I'm currently setting up different compose files for our applications, but I don't want to use features which are getting dropped in the next version and therefore preventing use from updating to a newer docker-compose file version.

@shin- shin- added the format/v3 label Jan 24, 2017

@shin-

This comment has been minimized.

Member

shin- commented Jan 24, 2017

At the moment, you should assume that the new depends_on syntax will not be ported to v3, as we have no current plans to do so.

I know that's not the answer a lot of people want, but I hope that helps give some clarity at least.

@mustafaakin

This comment has been minimized.

mustafaakin commented Jan 31, 2017

Can I ask why it is not in plans? I assume it would be very useful to do so.

@hsmade

This comment has been minimized.

hsmade commented Jan 31, 2017

It gives clarity, but doesn't explain. Can you elaborate on the why? And on the alternatives (if they exist)?
The depends_on gives us a really easy way to depend on a service, as opposed to handling it inside the container (which might mean wrapping a 3rd party image with some wait script and having to maintain that).

@dead10ck

This comment has been minimized.

dead10ck commented Jan 31, 2017

@shin- why did you implement it at all in 2.1, then? If people use it and come to depend on it, they will never be able to upgrade. With all due respect, that seems like very poor planning on your guys' part.

@brettmc

This comment has been minimized.

brettmc commented Jan 31, 2017

So what is the supported depends_on syntax for v3? https://docs.docker.com/compose/compose-file/#version-3 does not mention depends_on, and when I use docker-compose v1.10 to deploy an application neither the v2 or v2.1 depends_on flavours work for me in a v3 compose file...

@shin-

This comment has been minimized.

Member

shin- commented Jan 31, 2017

@mustafaakin

Can I ask why it is not in plans? I assume it would be very useful to do so.

@hsmade

Can you elaborate on the why? And on the alternatives (if they exist)?

From moby/moby#30404 (comment)

depends_on is a no-op when used with docker stack deploy. Swarm mode services are restarted when they fail, so there's no reason to delay their startup. Even if they fail a few times, they will eventually recover.


@brettmc

So what is the supported depends_on syntax for v3?

When using docker-compose, the supported syntax for v3 is the list syntax (similar to the one used for 2.0). If you're using docker stack deploy, dependencies will not be honored (see above for justification)


The version 3 format is the first step in moving away from the external docker-compose tool towards the integrated docker stack solution. The current implementation has its quirks which are being worked on. Support for the version 3 format in docker-compose is meant to help that transition. A lot of things have changed and improved in Docker since fig / Compose was first introduced, and that means a lot of the things that used to make sense have now become obsolete. docker stack is a fresh start using new concepts and sheds away some of the more unwieldy syntax and concepts, from volumes_from to depends_on.
If you have particular concerns about some of those changes, please report them over on the docker/docker repo where it is being actively iterated on.
If you are not yet ready to transition to Docker services and docker stack, feel free to keep using the v2 format. While it's reasonable to assume the project will be sunset at some point in the future, it will be announced well in advance. And after that the code will remain available and open source.

@shin- shin- closed this Jan 31, 2017

@wjv

This comment has been minimized.

wjv commented Feb 3, 2017

Thanks. Now it makes sense.

@mustafaakin

This comment has been minimized.

mustafaakin commented Feb 12, 2017

Swarm mode services are restarted when they fail, so there's no reason to delay their startup. Even if they fail a few times, they will eventually recover.

IMHO this is not a good approach. Not all services can properly detect the other services they depend are not ready, they try many times then fail, so container might die later. We still need to introduce entrypoint wrapper scripts, which is not very nice. Healthy-check dependence was very nice, but it just does not support swarm mode.

@Marvel77

This comment has been minimized.

Marvel77 commented Feb 14, 2017

Swarm mode services are restarted when they fail, so there's no reason to delay their startup. Even if they fail a few times, they will eventually recover.

Hey everyone!
Does it mean that if for example I have a service which runs and finishes its work very quickly (and should run only once by design) it will be started again and again and again... repeatedly?

@shin-

This comment has been minimized.

Member

shin- commented Feb 14, 2017

@Marvel77 By default, yes, but you can override that behavior using the deploy.restart_policy section: https://docs.docker.com/compose/compose-file/#restartpolicy

@Marvel77

This comment has been minimized.

Marvel77 commented Feb 15, 2017

@shin- Thank you very much!

@hsmade

This comment has been minimized.

hsmade commented Feb 15, 2017

@mustafaakin Actually, the best practice is (IMHO) to have fault tolerant connection to the services you depend on. For example, if you use a database, you might delay startup until it's responding. But how do you handle a restart of the database? Your app should be able to recover from this, and then you don't need to depend on 'depend_on' either.
In a sense, the deprecation is a good thing. These conveniences make us lazy.

@mustafaakin

This comment has been minimized.

mustafaakin commented Feb 15, 2017

@hsmade But almost all init (upstart,systemd) have depends on relationship. So it is not that being lazy, it is what makes sense. SSHD daemon does not start until you have your network device ready. I have no control over all the systems I have, yes I can make my systems be fault tolerant. But assume A depends on B. A takes a while to initialize, it is not very deterministic. So, how can you write a restart policy on B? restart forever? What if you do not want that?

This is a bigger problem than Compose, Compose just starts them. Swarm is what makes them run, so I think Swarm should handle this dependence on health relationship.

@hsmade

This comment has been minimized.

hsmade commented Feb 15, 2017

@mustafaakin I don't think you can compare running micro services in a containerized environment with the classic init systems. That doesn't really make sense. I think the idea of micro services is that they are self-contained, minimal entities. They have a very clear definition of their API, etc. They should not assume any status of their external world. Yes, it would need a database, but no, you can't assume it's there. As I tried to say, knowing that your dependency is there on startup is the least of your problems, handling it well when it goes away is more important, and should solve the former as well.

But then again, these are my thoughts on the matter, and I could be wrong ;)

@mustafaakin

This comment has been minimized.

mustafaakin commented Feb 15, 2017

Yeah, it does not make sense for microservices, but not everything is a microservice, I run Hadoop in a container. Docker is not for only microservices. As I said, its great for the environments I have control, but in the things that I do not have control, it requires wrapping the entrypoint of services. It was just solved with depends_on with healtcheck, I just think it would be great to have it in Swarm as well. Restarting contionously is just a lazy man's choice.

@ozlatkov

This comment has been minimized.

ozlatkov commented Feb 23, 2017

Guys,

I think there is a kind of mis-conception of "fault-tolerant" and a kind of "first-time initialization".
Agree that the approach sounds good enough for the first one, but the latter is a real pain.
I would imagine that not only myself, but generally one usually would need to start services in a particular order as they depend on each other more or less.

Having continuous restarts at initialization phase and waiting for the services at some point start in the proper order by themselves is like a nightmare - one cannot plan any "downtime" for the startup process should the worse happen. Not only the worse, but sometimes there are like say "maintenance" time when something needs to change and nobody would be able to predict how much time it would take for the system to actually start, because different services are restarted by swarm over and over again waiting for the proper order.
I've tried and waited to see how it will go with 7 containers and for 20 minutes swarm still hasn't figured it out and the whole service was still down.
So one cannot even state how much time it would take to bring back and restore the whole infrastructure as it's really unknown how long the initial startup would take.
Sounds absurd to me.

I don't think I can quite enter in "production" without such a predictability even though it's supposed to happen rarely (full down) - best never happen.
But exactly if it still happens I would be on the spot and will be challenged to restore ASAP and right at that time and pressure I would have absolute no idea how much time my containers would start just because they continue to restart themselves during startup.

@shin-

This comment has been minimized.

Member

shin- commented Feb 23, 2017

@ozlatkov This should really be posted on the docker/docker issue tracker, not here.

@ozlatkov

This comment has been minimized.

ozlatkov commented Feb 24, 2017

@shin- Thank you! I've moved it to docker/docker tracker:

moby/moby#31333

@jacek99

This comment has been minimized.

jacek99 commented May 31, 2017

I think it is really bad that the Docker team assumes Docker Swarm is used. Compose is supposed to a totally functional standalone tool.

We do however use Docker Compose a lot in our testing pipelines and such a fundamental REMOVAL of features (without giving a workable alternative) is really having a dramatic negative effects on your users.

I am currently reviewing a very ugly PR from my team members where they are trying to come up with all sorts of workarounds to this (since we rely heavily on this functionality), including staying on Compose 2.X for all eternity.

Docker is supposed to help us get to our goals, not make it more difficult by removing critical features many teams already rely on.

Please reconsider porting all of this into Docker Compose 3.

Much appreciated.

@shin-

This comment has been minimized.

Member

shin- commented Jun 1, 2017

For the 100th time now: There's no reason to use the v3 format if you don't intend to use swarm services.

@lukehau

This comment has been minimized.

lukehau commented Jun 15, 2017

Does that mean the team is committing to supporting the 2.X formats for those that are using compose as a standalone development tool?

@jacek99

This comment has been minimized.

jacek99 commented Jun 15, 2017

Exactly my question: is the Compose team committed to supporting the v2 format forever?

We cannot standardize on Docker Compose if the v2 format is scheduled for deprecation any time.

@shaunwarman

This comment has been minimized.

shaunwarman commented Sep 28, 2017

I feel like this forces all containers into an init container pattern, removes the docker restart policy and creates a hacky approach to startup order. Should it be assumed that >= v3 of compose will be moving in this direction to focus on stacks and creating application bundles? And if that's right, could you point me in the direction of how to maintain startup order in a docker stack? Is the wait-for-it.sh or dockerize the approach there?

@ntwrkguru

This comment has been minimized.

ntwrkguru commented Oct 6, 2017

What is the declarative equivalent of docker-compose.yml for a stack?

@VinceOPS

This comment has been minimized.

VinceOPS commented Nov 27, 2017

Hi guys, what's the best practice if I intend to use docker stack and get rid of docker-compose?

This seems to be the solution, but having sort of hacky scripts to init containers doesn't sound like a good practice. Does it?

Thanks.

@mustafaakin Thanks for your downvote! Very helpful ❤️

@hutchic

This comment has been minimized.

hutchic commented Nov 27, 2017

@VinceOPS I'm not sure about "best practice" but I've been using healthchecks and restart: always so it just cycles until it works ¯\(ツ)

@ailjushkin

This comment has been minimized.

ailjushkin commented Jan 15, 2018

@hutchic But as mentioned in the above conversation, it could not have an end date, some kind of a circular on-failure restart.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment