latest release not available in the official repo

[felicianotech]

I'm not sure if this belongs here or in https://github.com/docker/docker-ce-packaging. The official Debian/Ubuntu apt repository (referenced in docs) doesn't contain the latest release of Docker Compose, v2.6.1, which came out 22 days ago.

The package name, docker-compose-plugin, stops at v2.6.0. Any thoughts as to why?

[johnthagen]

I raised this issue in:

• Compose 2.5.1 not published as .deb or .rpm #9497

Not sure why, but not every Compose release is published to the Linux repositories. It would be very helpful if they were.

[felicianotech]

Agreed. Shouldn't be hard to automate in a CD process.

[felicianotech]

Docker Compose v2.7.0 is out now, and again not in the Apt repository. I would be more than happy to try and help get these builds automated if I could be pointed in the right direction.

[johnthagen]

I think this is the repository:

• https://github.com/docker/docker-ce-packaging

[johnthagen]

Though, perhaps this isn't 1 to 1 with the repos as this PR implies 2.7.0 was merged:

• [master] bump version of Compose to v2.7.0 docker-ce-packaging#721

[Paraphraser]

Same with 2.8.0.

I'm not trying to add pressure on anyone to do anything about this. I'm just trying to figure out what I should do as I build my own systems, so it would be nice to have some idea of a timeline for when this is likely to be fixed?

I used to download compose from the releases page but back in June I noticed that the "convenience script" had started including apt-get install docker-compose-plugin which installed 2.6.0. I thought "brilliant! - that means compose will now be updated via apt so I don't have to do the separate installation or keep track of updates - it will all happen automagically". But that excitement was obviously a bit premature. I'm now trying to figure out whether it's best to "sit tight and await developments" because this will be solved any day now, or go back to downloading the up-to-date versions from here because, behind the scenes, this issue is a lot harder to solve than an outsider might guess.

I'm also cross-linking to #9195 which mentions this same issue.

[felicianotech]

Thanks @Paraphraser for linking #9195. Reading through that issue, I think I might understand now what's going on.

Getting new releases of Docker Compose into the Apt repository doesn't happen in this repo. As @johnthagen mentioned, this happens in the repo https://github.com/docker/docker-ce-packaging. PRs from that repo (link this one docker/docker-ce-packaging#721) assign the latest Docker Compose version, but that doesn't actually get it published. It seems that the docker-compose-plugin package that is assigned only actually gets published when a new release of Docker CLI itself goes out (ref: #9195 (comment)).

So basically, Docker Compose will only be updated in the Apt repo when Docker CLI gets published there. When that does happen, it will be the Docker Compose version listed in https://github.com/docker/docker-ce-packaging/blob/master/common.mk#L43.

---

This is still a problem, and the comment I posted above suggest they know about it and are trying to unlink the two, but I'm not sure what that timeline will be.

[nickjj]

Is there something "interesting" that's going on with the docker-compose-plugin package that would make it not behave like other packages in the system?

For example on Ubuntu 22.04 LTS I can't apt pin docker-compose-plugin with:

Package: docker-compose-plugin
Pin: version 2.5.0*
Pin-Priority: 600

No matter what I do it always installs 2.6.0. It's like it ignores the pin.

But 2.5.0 is available based on:

$ apt policy docker-compose-plugin
docker-compose-plugin:
  Installed: 2.6.0~ubuntu-jammy
  Candidate: 2.6.0~ubuntu-jammy
  Version table:
 *** 2.6.0~ubuntu-jammy 500
        500 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages
        100 /var/lib/dpkg/status
     2.5.0~ubuntu-jammy 600
        500 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages
     2.3.3~ubuntu-jammy 500
        500 https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages

I've pinned docker-ce using this same strategy and it works fine.

[nickjj]

This issue occurs without docker-compose-plugin being installed beforehand. It actually turned out to be a bug with Ansible 2.13 not respecting pins.

[th0ger]

Same here, only 2.6.0 available, despite latest release is up to 2.10.2.

$ apt-cache madison docker-compose-plugin
docker-compose-plugin | 2.6.0~ubuntu-jammy | https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages
docker-compose-plugin | 2.5.0~ubuntu-jammy | https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages
docker-compose-plugin | 2.3.3~ubuntu-jammy | https://download.docker.com/linux/ubuntu jammy/stable amd64 Packages

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy

$ cat /etc/apt/sources.list.d/docker.list
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu   jammy stable

[rusagent]

I have the same problem and actually just realized it now.

apt-cache madison docker-compose-plugin
docker-compose-plugin | 2.6.0~debian-buster | https://download.docker.com/linux/debian buster/stable amd64 Packages
docker-compose-plugin | 2.5.0~debian-buster | https://download.docker.com/linux/debian buster/stable amd64 Packages
docker-compose-plugin | 2.3.3~debian-buster | https://download.docker.com/linux/debian buster/stable amd64 Packages

lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

cat /etc/apt/sources.list.d/docker.list
deb [arch=amd64] https://download.docker.com/linux/debian buster stable

[th0ger]

@rusagent as a workaround I installed the plugin manually
https://docs.docker.com/compose/install/linux/#install-the-plugin-manually

[johnthagen]

It looks like docker-compose-plugin 2.10.2 was just published: https://download.docker.com/linux/ubuntu/dists/bionic/pool/stable/amd64/

[ndeloof]

As long as system packages are managed by docker/docker-ce-packaging and we don't get release cycles aligned there's nothing we can do to enforce latest compose release is available by this distribution channel. Maybe in the future things will improve but this isn't under compose team responsibility. In the meantime it's easy to just download the platform binaries from github release page as a workaround, so I'm closing this issue.

[jflattery]

@ndeloof

Is the Compose team not part of Docker Inc.? Also, as Docker Compose is an official Docker project, I was under the impression that Docker Inc. would have some level of control over the Docker apt repository (for example, https://download.docker.com/linux/ubuntu). Could you please clarify if my understanding is incorrect?

[kchawla-pi]

As long as system packages are managed by docker/docker-ce-packaging and we don't get release cycles aligned there's nothing we can do to enforce latest compose release is available by this distribution channel. Maybe in the future things will improve but this isn't under compose team responsibility. In the meantime it's easy to just download the platform binaries from github release page as a workaround, so I'm closing this issue.

In addition to what @jflattery said, it is also the Compose team's responsibility to have appropriate documentation.

The official documentation for Docker Compose installation on Linux specifically encourages setting up and using the apt repository as the preferred method.

And nowhere is it mentioned that the apt repository version is not updated regularly and can be severely outdated and the version on GitHub releases is the only way to get the newer versions.

But the compose features and help documentation always point to the latest few versions by default.

If official documentation endorses the apt repo, doesn't mention any caveats right up front right where these instructions are, and then the official help docs refer to the newer versions, one would automatically assume, since this is what usually should and does happen with most other softwares, that doing apt upgrade of docker products will bring them all to current versions.

So here are Docker users, everything installed and working just fine, busy in their work day with deadlines and all, apt upgrading docker once every 6 months because that should be enough time to iron out new release issues. And everything is working fine because the compose files aren't using relatively newer features yet; then they try using a newer feature because hey, it's a useful one and can save hours of work in the long run, when bam! The new compose file does not work, no idea why so they spend some time digging into it until they find that the compose version is way behind what the docs are for, apt upgrade be damned.

[ndeloof]

I understand the frustration but this is a larger organization-level discussion to have to refactor the package repository distribution pipeline. docker/docker-ce-packaging would be the best place to track required changes for this to take place

[kchawla-pi]

And like @jflattery said, this isn't Oh! Canonical keeps changing their apt thingamajig which keeps breaking our builds and it is not in our control!

No, this is Docker Engine team not talking to Docker Compose team. Two foundational

This issue is nearing its second anniversary; in all that time, nobody thought to corner Mike at the company retreat and say, Hey! maybe we should keep our versions copacetic, yeah?

[kchawla-pi]

@ndeloof The Compose team is telling us there is nothing they can do? The Compose team cannot update its installation instructions to recommend just installing from the GitHub releases, or at the barest minimum of effort, add the caveat with the repo installation method documentation?

[kchawla-pi]

I understand the frustration but this is a larger organization-level discussion to have to refactor the package repository distribution pipeline. docker/docker-ce-packaging would be the best place to track required changes for this to take place

It has been 2 years! This is a mainstream product from a "large organization" used by million!
Wasn't "Excellent Development Experience" the new Docker motto?

[ndeloof]

I only can agree this must be fixed, but unfortunately this repo is not the right place to have impact on this as the compose team has no impact on the release pipeline, which was initially designed for docker engine.
Please refer to issue you created on https://github.com/docker/docker-ce-packaging for this purpose (also applies to buildx by the way)

[kchawla-pi]

What about changing the documentation for Docker Compose?

[kchawla-pi]

Please refer to issue you created on

Also, this is messed up, telling me to refer to the issue I created 10 minutes ago, that you acknowledge I created 10 minutes ago.

[ndeloof]

my comment was for anyone joining this conversation, not just you (obviously)

[kchawla-pi]

Alright, any answers to this? Will the compose team discuss this?

What about changing the documentation for Docker Compose?

[ndeloof]

What about changing the documentation for Docker Compose?

docs already includes instructions for both installation approaches:
https://docs.docker.com/compose/install/linux/

Please also note, while the latest version may not be immediately available on download.docker.com, the engine release cycle has increased the lack of release sync has a lower impact now. 2.26.1 is for example available just now (https://download.docker.com/linux/ubuntu/dists/noble/pool/stable/amd64/) so this isn't as bad as it used to be one year ago.

[Paraphraser]

I'd like to offer my very strong support to the people who work on docker-compose. They do excellent work and deserve our appreciation, tolerance and respect.

---

@kchawla-pi if you are two years behind then your installation of docker-compose is almost certainly incorrect. It's probably not your fault. One very common reason is having followed instructions on the web or in a YouTube video - and those instructions were actually wrong.

It boils down to some history, and the difference between the docker-compose and docker-compose-plugin packages.

Many instructions tell you (in part) to install:

$ sudo apt install docker-compose

That is wrong. It results in an ancient version of docker-compose which can never be upgraded by apt.

Worse, when you use apt to install the ancient version of docker-compose, docker is forcibly downgraded and it, too, can never again be upgraded by apt.

If you install Docker on a brand new Linux system you should use the official "convenience script". Despite it appearing under the Ubuntu heading, the script works properly on Debian and Raspberry Pi OS. The script installs docker and docker-compose-plugin, plus several other packages.

The practical effect of running the convenience script is the plugin form works. For example:

$ docker compose up -d

One unfortunate side-effect is that the traditional command form no longer works. For example, this fails:

$ docker-compose up -d

The difference is the hyphen between docker and compose.

The easiest way to fix this problem is with a symbolic link, then both the plugin and command forms work.

There are some other things the "convenience script" does not do, including adding the user who will be running docker commands to the docker group. Doing that avoids the need to use sudo.

Although the documentation I am about to point you to is tailored to a specific environment (IOTstack), you will find some information here about repairing an existing system so you have an up-to-date docker and docker-compose-plugin.

I am reasonably sure that the scripts in the helpers directory will work on any Debian or derivative system.

The 04_setup.sh mentioned assumes the predecessor scripts (01…03) have already been run so it may need some tinkering to run standalone. The core of it is:

$ curl -fsSL https://get.docker.com | sudo sh
$ sudo ln -s "/usr/libexec/docker/cli-plugins/docker-compose" "/usr/local/bin/docker-compose"
$ sudo /usr/sbin/usermod -G docker -a $USER

Just keep in mind that you are unlikely to be able to run it successfully unless your existing docker and docker-compose have been removed - completely - and that isn't always as simple as an apt purge. The uninstall scripts are better.

Now, for the record, here's my own system:

That's fully up-to-date and was all done with routine apt update/upgrade. No smoke and mirrors. Nothing up my sleeve.

Finally, it is true that what you can get from apt upgrade sometimes lags behind what is available on the docker-compose releases page on GitHub. Personally, I think the lags are totally reasonable and acceptable. I like to think any delays between something appearing on GitHub and making it into the apt repositories is likely explained by testing regimes and, frankly, I'd rather other people do as much testing as possible before the package arrives on my system. But that's just me.

However, if you really want to use the most-recent release then the helpers directory also contains install_docker-compose.sh which downloads and installs whatever version you specify from GitHub (ie completely bypasses apt).

I'd really appreciate it if you could add comments to this issue to let me know whether what I've written above has helped you to resolve your problem.

[kchawla-pi]

@Paraphraser none of what you said applies to me or anything I wrote.

My installation isn't two years old, this issue was opened 2 years ago.

I didn't use YouTube. I used official documentation.

We all appreciate the software and the team that builds it and we will also critique it. The two aren't mutually exclusive. In fact if we didn't appreciate it we wouldn't bother filling issues here.

[kchawla-pi]

@Paraphraser all I want at minimum is for the official documentation to mention this caveat and point to GitHub releases page as the workaround.

Ideally that should be made the default not apt.

This is one of many examples:

This is from just a few months ago, when the compose documentation mentioned the new include directive which was supported by Docker Compose installed via Docker Desktop for Mac and Windows immediately but wasn't available with the Docker compose installed via apt following the official installation instructions for many months after.

And no, this is not one specific thing I want a solution to, it is just one example that I clearly remember right now.