Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support compose version > 2 #2054

Closed
dazinator opened this issue Jul 19, 2018 · 60 comments
Closed

support compose version > 2 #2054

dazinator opened this issue Jul 19, 2018 · 60 comments

Comments

@dazinator
Copy link

@dazinator dazinator commented Jul 19, 2018

Is your feature request related to a problem? Please describe.

I tried to deploy a stack from the stacks tab and pasting in contents of a docker-compose.yml file.
Unfortunately the yml in question is using version 2.4 of the compose spec as it needs things added in that version.

I noted the message in the UI that says only compose version 2 is supported. True to that warning, I got an error when attempting to deploy my 2.4 version.

Would you consider adding supporting for the latest version?

@deviantony

This comment has been minimized.

Copy link
Member

@deviantony deviantony commented Jul 19, 2018

Yes, we'll consider this as an enhancement. It implies to upgrade libcompose to support compose > 2

@f3l1x

This comment has been minimized.

Copy link

@f3l1x f3l1x commented Jul 28, 2018

Thanks. I'm looking forward to it.

@flothemaker

This comment has been minimized.

Copy link

@flothemaker flothemaker commented Aug 11, 2018

If you upgrade libcompose, would it be possible to deploy compose version >3 from portainer?

@deviantony

This comment has been minimized.

Copy link
Member

@deviantony deviantony commented Aug 11, 2018

@braaccckkk I believe this is the way to go to support version > 2

@mariolopjr

This comment has been minimized.

Copy link

@mariolopjr mariolopjr commented Aug 12, 2018

@deviantony That would be amazing! I didn't realize just how many version 3 features I used (even version ~2.4 like HEALTHCHECK, etc.). Thanks to all for the wonderful work! :)

@krysennn

This comment has been minimized.

Copy link

@krysennn krysennn commented Aug 20, 2018

Strange...
I'm using docker-compose 3.6 version with Portainer and it works very well.

@deviantony

This comment has been minimized.

Copy link
Member

@deviantony deviantony commented Aug 20, 2018

@krysennn You can deploy a Swarm stack (docker stack deploy equivalent) version > 3, but we only support Standalone stacks (docker-compose equivalent) version 2.

@krysennn

This comment has been minimized.

Copy link

@krysennn krysennn commented Aug 20, 2018

Oh! You're right, sorry i'm confused :)

@MicahZoltu

This comment has been minimized.

Copy link

@MicahZoltu MicahZoltu commented Sep 1, 2018

Where can I get more information on the difference between Swarm Stacks and Standalone Stacks? I use docker-compose files to manage my deployments and make them portable across tools, but in Portainer UI I don't see a way to deploy a Swarm Stack, only the v2 stacks. I'm really struggling with the limitations of v2.0 docker-compose files (in particular, networking related) and would like to figure out how to use a newer version of docker-compose files (e.g., v3 or at least v2.4).

@ncresswell

This comment has been minimized.

Copy link
Member

@ncresswell ncresswell commented Sep 1, 2018

@MicahZoltu

This comment has been minimized.

Copy link

@MicahZoltu MicahZoltu commented Sep 1, 2018

Not that I know of. :) I have a host (Ubuntu), I installed Docker CE on it, I ran the Portainer container. If that doesn't result in me having a Swarm cluster then no, I don't have one.

Feel free to redirect me to some documentation if I'm just missing some critical piece of a puzzle here. I am using Portainer to manage a single docker host, not a complex multi-host orchestration system. I just want access to newer versions of docker-compose is all.

@deviantony

This comment has been minimized.

Copy link
Member

@deviantony deviantony commented Sep 1, 2018

Hi @MicahZoltu

A simple docker swarm init command will allow you to create a simple one node Swarm cluster and give you access to Swarm features.

Once your host is in Swarm mode, Portainer will auto-detect it and you'll be able to use Compose v3+ to deploy stacks.

@josephduchesne

This comment has been minimized.

Copy link

@josephduchesne josephduchesne commented Nov 19, 2018

These aren't exactly equivalent since there are a decent number of docker-compose options that docker swarm ignores (--device, --privileged, etc.). Support for docker-compose 3.X would be really nice!

@deviantony deviantony changed the title support compose 2.4 support compose version > 2 Jan 3, 2019
@lsascha

This comment has been minimized.

Copy link

@lsascha lsascha commented Jan 22, 2019

Just noticed this limitation for non-stack setups after preparing templates etc... really annoying :(

@ncresswell

This comment has been minimized.

Copy link
Member

@ncresswell ncresswell commented Jan 22, 2019

@joseluisq

This comment has been minimized.

Copy link

@joseluisq joseluisq commented Jan 22, 2019

More info Libcompose Unmaintained

@lsascha

This comment has been minimized.

Copy link

@lsascha lsascha commented Jan 22, 2019

Thanks. i now just initialized a single node swarm to use newer compose file versions.
I was just baffled and unaware that this limitation existed because i used it successfully on a swarm setup before,

Is libcompose not used for swarm setups?

@deviantony

This comment has been minimized.

Copy link
Member

@deviantony deviantony commented Jan 22, 2019

@lsascha Portainer uses libcompose behind the scene to deploy stacks on standalone Docker hosts (as you would do when using the docker-compose tool) and uses the Docker CLI binary to deploy stacks on Swarm environments (equivalent of docker stack deploy ... command).

@lsascha

This comment has been minimized.

Copy link

@lsascha lsascha commented Jan 22, 2019

thx for the explanation. Now another stupid question. Why is the official docker-compose CLI tool not used to deploy to standalone Docker hosts? ;)

@deviantony

This comment has been minimized.

Copy link
Member

@deviantony deviantony commented Jan 22, 2019

Mainly because the Portainer image aims to be a lightweight multi-platform image that can be deployed on any OS/arch and that the compose binary is only available for linux and windows amd64.

We would either need to bundle python into our Portainer images or create our own unofficial build pipeline for compose on the different platforms we support (assuming it's possible).

Overall, interfacing with an API/library is much easier/has less constraints than interfacing with a binary. Although I'm aware that our possibilities are limited in this context.

If we decide to limit our os/arch target scope we would be able to switch to bundling/using the official compose binary. A potential way to do it might be to disable stack management on the platform where the binary is not available.

Another way of doing it would be to create an extension that would bring enhanced compose support for these platforms.

@doubled1

This comment has been minimized.

Copy link

@doubled1 doubled1 commented Feb 3, 2019

What about the other way around? Use the binary if available and fall back to what it uses right now if it isn't?

@deviantony

This comment has been minimized.

Copy link
Member

@deviantony deviantony commented Feb 3, 2019

That could be a potential way of working around this too.

@alphaDev23

This comment has been minimized.

Copy link

@alphaDev23 alphaDev23 commented Feb 24, 2019

Very frustrating. Deploying stacks via docker-compose 3.x version works well in Portainer 1.17.1. Furthermore, 3.x fully supports deploying swarm stacks per the docker-compose reference: "A good place to start is the Getting Started tutorial which uses version 3 Compose stack files to implement multi-container apps, service definitions, and swarm mode." See also https://docs.docker.com/compose/compose-file/

@rcdailey

This comment has been minimized.

Copy link

@rcdailey rcdailey commented Mar 12, 2019

Thank you for explaining. Does that mean that going forward, even for docker compose, doing services without swarm is not getting long term support? In other words, is it better to eventually move to swarm even if I only have 1 machine and do not need replication/etc and stuff?

@ncresswell

This comment has been minimized.

Copy link
Member

@ncresswell ncresswell commented Mar 12, 2019

@mersadk

This comment has been minimized.

Copy link

@mersadk mersadk commented Mar 19, 2019

I've switched to single node swarm setup but now I can't access any of my containers. I expose port 55500 but can't access it with localhost:55500. Do I need to run anything else besides docker swarm init?
When I manualy change network of container to nat, it works.
I'm running it on Windows Server 2019 with Docker Enterprise.

@MicahZoltu

This comment has been minimized.

Copy link

@MicahZoltu MicahZoltu commented Mar 19, 2019

@mersadk This probably isn't the right place to troubleshoot swarm problems, but in case it helps:

I think by default swarm has an ingress network that it attaches containers to and when you expose a port it binds to that network, thus binding your service to that network which should make it accessible from localhost. However, your stack file may specify a different network, or perhaps that network is setup differently on your host (likely if you have multiple networks on the host I would imagine).

@itsconquest

This comment has been minimized.

Copy link
Member

@itsconquest itsconquest commented Mar 24, 2019

As mentioned in previous comments above, Portainer can only support compose v3 or newer in a Swarm environment due to Docker depreciating libcompose.
I will now close this issue.

@js-mode

This comment has been minimized.

Copy link

@js-mode js-mode commented May 16, 2019

No luck on getting Portainer to work with v3? We were hoping to be able to pass in environment variables through a file or .env type set up which we can't seem to get with portainer in a single stack type deployment. Are there other work arounds in place to possibly pass in secrets through Portainer that don't use the ENV format?

@ncresswell

This comment has been minimized.

Copy link
Member

@ncresswell ncresswell commented May 16, 2019

@rcdailey

This comment has been minimized.

Copy link

@rcdailey rcdailey commented May 16, 2019

@Ryonez

This comment has been minimized.

Copy link

@Ryonez Ryonez commented Jun 23, 2019

Shouldn't we be looking at a library with support then?

I use docker on unRAID. As far as I know, that doesn't use docker swarm. I've also installed docker-compose and have been using version 3 files.

Even reading above I don't see why portainer can't do this as I have a working non swarm environment.

@ncresswell

This comment has been minimized.

Copy link
Member

@ncresswell ncresswell commented Jun 23, 2019

@HWiese1980

This comment has been minimized.

Copy link

@HWiese1980 HWiese1980 commented Aug 29, 2019

How's the status of this issue? I see it's closed. Is >2 support already integrated?

@HWiese1980

This comment has been minimized.

Copy link

@HWiese1980 HWiese1980 commented Aug 29, 2019

A pity, I could really put it to good use right now... unfortunately I don't have a swarm cluster at hand, only a single PC where the stack shall be deployed on. Well, looks like I'll have to do it by hand.

@ncresswell

This comment has been minimized.

@HWiese1980

This comment has been minimized.

Copy link

@HWiese1980 HWiese1980 commented Aug 29, 2019

I'll give it a shot. Thanks!

@HWiese1980

This comment has been minimized.

Copy link

@HWiese1980 HWiese1980 commented Aug 29, 2019

Is there a list of compatible compose versions? Now it's complaining about an unsupported docker compose version 2.3

(I initialized a single swarm node on my PC)

@HWiese1980

This comment has been minimized.

Copy link

@HWiese1980 HWiese1980 commented Aug 29, 2019

And on a single Swarm node, now that I have initialized it?

@KopfKrieg

This comment has been minimized.

Copy link

@KopfKrieg KopfKrieg commented Aug 30, 2019

Single node swarm cluster is massively beneficial.

Doesn't it also have drawbacks?

@MicahZoltu

This comment has been minimized.

Copy link

@MicahZoltu MicahZoltu commented Aug 30, 2019

Doesn't it also have drawbacks?

Not that I have run into now that I'm fully migrated. I think the reason docker didn't just automatically move everyone to swarm is that there are breaking changes between non-swarm and swarm docker. All of my single-node docker hosts are swarm clusters of 1.

@bjmi

This comment has been minimized.

Copy link

@bjmi bjmi commented Sep 9, 2019

Doesn't it also have drawbacks?

Privileged mode is not supported in docker swarm.

@ncresswell

This comment has been minimized.

Copy link
Member

@ncresswell ncresswell commented Sep 9, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.