Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
compose "version 3" is not a version #4693
@shin- It is completely understandable that the team would want to create an optimal compose file for use with docker swarm and tying in closely with the docker stack. Personally I'm a fan of docker swarm and I hope for elegant compose tooling to use with it.
@shin- Here #4513 (comment) you talk about compose v2 continuing support, but also how you hope the docker stack will take over most usecases then this #4673 (comment) seems to imply that the community can just take on updating compose.
Moreover, compose v3 is marked as the "current, recommended" format. Likewise there are steps on "upgrading" https://docs.docker.com/compose/compose-file/compose-versioning/#upgrading
THIS DOES NOT INSPIRE CONFIDENCE
Either option would allow everyone to use the recommended version of docker-compose.
In any case it would be nice to have a more clear explanation of dockers plans for compose and potentially when the community should expect to support non-docker stack functionality on our own.
Agreed. Talking about functionality, I miss
The deploy section in v3 documentation does mention that, but the point is that upgrading to the latest and greatest version brings the problem of some things not being supported anymore. One upgrades all the parameters and then finds that the latest version was intended to swarm users.
If no solution will happen prior to v3+, maybe a flag/note in the Compose File Reference stating that latest version is intended for swarm only, and not recommending for general usage.
Is there any reason why
I understand that it doesn't make sense with some keys, e.g.
Based on this, I believe, most options can be supported, and only a few that cannot should result in warnings. Would appreciate further discussion/opinions on this.
If there will be an agreement that Compose indeed can and should support those (and how exactly), I can try to work on a PR.
I'm going to try and answer that to the best of my ability. Please understand that I don't necessary hold all the information here. I'll try to see if I can have people responsible for
First off, a point of semantics: "Docker Swarm" and "Swarm Mode" are technically 2 different things. The former is a product that has existed for some time now and is separate from the Docker Engine. The latter is a feature of the Docker Engine that was added in 1.12.0. Functionally they both fulfill the same role (clustering and orchestration), but in practice they function differently. 
As far as support of v2 is concerned, nothing indicates right now that we are going to discontinue or stop maintaining it anytime soon. If and when sunsetting it is decided, I trust Docker will announce this well in advance so there will be ample time for the then-users to decide whether they're ready to switch over to v3, or if they need to fork and maintain a community version of
Compose v3 is the "current, recommended format" because it is the format we want new, oncoming users to adopt, as it is specifically designed for distributed applications scaling over a Swarm cluster. If you consider this is the goal of most application developers in the long run, it makes a lot of sense to recommend the format that achieves that painlessly rather than the one that attempts it, but struggles due to its technical limitations .
Realistically, we are not going to see major changes to the v3 format that you suggest, because it would straight-up break versioning. I'm not sure I got your point about shared libraries, but please consider that
Regarding resources: I think the typical use-case we have in mind internally is that people using Compose v3 right now probably want to benefit from the simplicity of
@drdaeman Those options have a specific meaning in the context of Swarm services. It would be a mistake to try and map those to seemingly equivalent container properties, or worse, try to re-implement Swarm logic in Compose as you suggest for
 If you're still confused, "Swarm mode" is essentially Docker's second attempt at getting clustering right, in a more integrated fashion, building upon the lessons learned with "Docker Swarm" (including re-using some of the code and concepts).
@shin- Thank you for your detailed reply!
Oh, no, sorry, I must've clarified it better. I've assumed that Compose - just like Swarm - also has a proper notion of services, with the main difference that Compose services are completely external to Docker and are managed solely by Compose, with no other piece of software being aware about those.
So I wrote not about containers, but about services (given that a single Compose service may consist of multiple containers, e.g. with
That is, unless
I don't see why it would be a mistake to implement some comparable Swarm Mode features (when possible and when makes sense) in Compose as well. Can you please clarify this a bit further?
I just don't see anything wrong if
Unfortunately, no. Fun thing is, I'm currently doing things in exactly opposite manner. I'm developing and testing stuff with
Swarm Mode lacks some features at the moment, but I'm planning to switch as soon as all the relevant issues I'm tracking are resolved. My desire for Compose to support v3.x deploy options comes from desire to maintain
We use docker-compose in dev and for CI right now (and actually don't deploy docker containers at all) and the main change that is preventing us from moving to v3 is not being able to set
Even though i see that some that some of the initial concerns have been covered by @shin- - the lack of volumes_from has not been covered.
I understand that most usage of volumes_from has been wrong, should have been simple named volume usage and thats basically it. But there is one scenario which is very common for docker stacks and that scenario has no longer an option to be implemented at all.
Assume you have this general case: 2 services, one "app" and one "httpd(proxy)". What we want is to have httpd serve static content and offer a upstream server for dynamic content ( ruby/php/java whatever ). So the very usual way of implementing the reference web application.
At this point, my question is, do we agree, that the above is the reference web-app application, language agnostic ( well at least we take out client side only application from this, but thats not like those are the common ones ).
App includes our code under
We do this, because we want to access this folder on
Ok, i guess some people will jump onto that with the argument - use a named volume instead - and its done? No it is not. The problem is the upgrade. We want the codebase to be part of the APP image, and if the app image has been rebuild due to codebase changes, we want to obviously distribute this.
When you do a
When you will use a named volume for /var/www - it will not.
And there is no way to work around this beside starting to do crazy things like removing some "named volumes" yourself on upgrade.
And there is more to that - your forced people to adopt v3 since it felt like it is the successor of of v2, similar to times when you introduced v2 over v1. And now the all start using named volumes in this scenarios and they will all be surprised by the fact, there applications will run on old code bases. So this is a huge inconvenience, waiting in the dark to bite your ass.
The upgrade mode "docker-compose pull / down / up" is common for all orchestrators i know - thus they will no longer work and people are left without an alternative.
Bottom line, removing volumes_from was maybe planned to be in a positive manner to stop people abusing it - but you removed a important tool which know completely blocks us ( and a lot more, as they will find out ) to use v3 ever.
We now have no way of Setting Memory limits (mem_limit) for containers. Currently we don't have a need to run swarms, as we only have one host in production, so we have to stick with version2 even though version 3 is now the preferred and recommended approach?
I still don't understand the rational for this.
I think this is part of the disconnect. I started looking today to use compose for CI testing against a clustered version of our app. Seeing 3 is the newest and recommended I started there only to finally discover I actually have to use v2 to be able to define # of replicas, unless I want to deploy to swarm.
I think there is a disconnect between those who want to only use docker compose for development and its direction as a tool for use with swarm.
This is still an issue and confusing to users.
I spent lots of hours learning and adapting all my compose files (20+) to v3 thinking that this is the way moving forward, to keep things updated, and to take advantage of the latest features. I now find I cannot limit resources and I have to convert everything back to v2, including applications running in production.
I use docker compose extensively for development environments, and I also run multiple applications that do not need scaling even in production. I just need to organize my containers, I do not want to have to learn and maintain swarms neither do I need for mentioned use cases, it's an overkill.