[BUG] instructions don't quite work under Unbuntu Linux

[goblimey]

Miscellaneous first checks

• I checked that all ports are open and not blocked by my ISP / hosting provider.
• I know that SSL errors are likely the result of a wrong setup on the user side and not caused by DMS itself. I'm confident my setup is correct.

Affected Component(s)

readme.md

What happened and when does this occur?

I installed the latest docker-compose as suggested and downloaded the docker-compose.yml.  It specifies version 3.8 at the top.  I ran the first docker command to create an email address, then ran the second to bring up the server permanently:


docker-compose up -d mailserver

I get an error saying that the version in the docker-compose.yml is wrong (see log)

Cranking the version down to 3.5 fixes the error.

### What did you expect to happen?

```Markdown
I guess this issue may be caused by different versions in different operating systems - I presume that the supplied docker-compose.yml has been tested and works in some environments.  It would be useful if it could be general-purpose.  If that's not possible then some guidance on this issue would be handy.

How do we replicate the issue?

1. install the mail server on a clean ubuntu server.
2. docker-compose up -d mailserver
3.
...

DMS version

latest

What operating system is DMS running on?

Linux

What instruction set architecture is DMS running on?

x86_64 / AMD64

What container orchestration tool are you using?

Docker Compose

docker-compose.yml

version: '3.8'

services:
  mailserver:
    image: docker.io/mailserver/docker-mailserver:latest
    container_name: mailserver
    hostname: mail
    domainname: goblimey.com
    ports:
      - "25:25"
      - "143:143"
      - "587:587"
      - "993:993"
    volumes:
      - ./docker-data/dms/mail-data/:/var/mail/
      - ./docker-data/dms/mail-state/:/var/mail-state/
      - ./docker-data/dms/mail-logs/:/var/log/mail/
      - ./docker-data/dms/config/:/tmp/docker-mailserver/
      - /etc/letsencrypt:/etc/letsencrypt
      - /etc/localtime:/etc/localtime:ro
    environment:
      - SSL_TYPE=letsencrypt
      - ENABLE_SPAMASSASSIN=1
      - SPAMASSASSIN_SPAM_TO_INBOX=1
      - ENABLE_CLAMAV=1
      - ENABLE_FAIL2BAN=1
      - ENABLE_POSTGREY=1
      - ENABLE_SASLAUTHD=0
      - ONE_DIR=1
      - DMS_DEBUG=0
    cap_add:
      - NET_ADMIN
      - SYS_PTRACE
    restart: always

Relevant log output

ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.

Other relevant information

No response

What level of experience do you have with Docker and mail servers?

No response

Code of conduct

• I have read this project's Code of Conduct and I agree
• I have read the README and the documentation and I searched the issue tracker but could not find a solution

Improvements to this form?

The "what level of experience" dropdown shows me only four options, all for people with little experience.  (It may be that there are more but the browser is not displaying them.  This is using Chrome under Ubuntu linux.)

[georglauterbach]

@casperklein I bet you that this is related to the version of docker-compose that @goblimey is using. This field was originally present but removed in #2246 😆 😆 😆 https://en.wikipedia.org/wiki/Murphy%27s_law

@goblimey please show us the output of docker-compose --version.

[casperklein]

@georglauterbach Yeah, because it should not matter, which docker-compose version you are using. We have no special syntax that only works in a specific compose file version. Our example should be fine with all docker-compose versions.

The version field is just informational and deprecated for some time now.

Top-level version property is defined by the specification for backward compatibility but is only informative.
A Compose implementation SHOULD NOT use this version to select an exact schema to validate the Compose file, but >prefer the most recent schema at the time it has been designed.

https://github.com/compose-spec/compose-spec/blob/master/spec.md#version-top-level-element

@goblimey Please remove the version tag all together and see if it works.

[goblimey]

Aha! It's a silly path issue. I've downloaded the latest version of docker, but I'm not using it.

The README advises me to go to https://docs.docker.com/compose/install/ to ensure that I have the latest docker-compose. That document has different sections for different OS, and the one for linux advised me to download it using curl into /usr/local/bin:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

However, on my system, I already have version 1.17 in /usr/bin:

$ which docker-compose
/usr/bin/docker-compose

You might imagine that I can just use the curl to overwrite /usr/bin/docker-compose, but I seem to end up with a binary that runs without error but does nothing.

However, I can run the new version by hand:

$ /usr/local/bin/docker-compose up -d mailserver
Recreating mailserver ... done
simon@goblimey1:~/docker-mail$ docker ps
CONTAINER ID   IMAGE                                 COMMAND                  CREATED         STATUS         PORTS                                                                                                                                                                                         NAMES
93fa1cd0430b   mailserver/docker-mailserver:latest   "/usr/bin/dumb-init …"   7 seconds ago   Up 6 seconds   0.0.0.0:25->25/tcp, :::25->25/tcp, 110/tcp, 0.0.0.0:143->143/tcp, :::143->143/tcp, 0.0.0.0:587->587/tcp, :::587->587/tcp, 465/tcp, 995/tcp, 0.0.0.0:993->993/tcp, :::993->993/tcp, 4190/tcp   mailserver

So now I have a working mail server.

And I can bring it down cleanly:

$ /usr/local/bin/docker-compose down
Stopping mailserver ... done
Removing mailserver ... done
Removing network dockermail_default
$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

So your README is fine. It's just more difficult than it should be to get a working docker-compose.

[mathematicalmichael]

fwiw: I needed to add version: '3' to the top of docker-compose.yml

[gdm85]

fwiw: I needed to add version: '3' to the top of docker-compose.yml

This version: '3' line on top is necessary when using Debian 11, otherwise it will fail and not start; alternatively if using the Debian LTS version of Docker is not supported I think it could be mentioned in https://docker-mailserver.github.io/docker-mailserver/latest/usage/ since LTS versions are widely used.

[georglauterbach]

The Docs will soon contain an information about us using Docker Compose V2. This will then align everyone on what our docs assume to be installed on the host.

[polarathene]

alternatively if using the Debian LTS version of Docker is not supported

I found no information about such an LTS version of Docker?

Debian's own docs for Docker also direct the user to upstream to install. Older releases of Docker Engine have various issues that have been resolved since, especially with v23 release this year updating BuildKit from 2020 release. Likewise with Docker Compose changes that have moved from the python plugin to Go lang rewrite as V2, which now has a versionless schema in compose.yaml.

[gdm85]

I found no information about such an LTS version of Docker?

Debian's own docs for Docker also direct the user to upstream to install. Older releases of Docker Engine have various issues that have been resolved since, especially with v23 release this year updating BuildKit from 2020 release. Likewise with Docker Compose changes that have moved from the python plugin to Go lang rewrite as V2, which now has a versionless schema in compose.yaml.

This is the package that I used: https://packages.debian.org/bullseye/docker-compose (from universe/admin)

[polarathene]

Nothing about that is LTS, it's just the package version shipped with Debian Bullseye. That is a drawback with choosing to use a distro like Debian. They provide slower pace of changes and releases, but this does sense of "stable" does not equate to better when it misses out on various improvements and bug fixes.

Version 1.25 was released in Nov 2019. I would suggest following the debian docs own endorsement of installing newer versions as documented by Docker docs for Debian. You are already 3 years behind in development there.

[gdm85]

Nothing about that is LTS, it's just the package version shipped with Debian Bullseye. That is a drawback with choosing to use a distro like Debian. They provide slower pace of changes and releases, but this does sense of "stable" does not equate to better when it misses out on various improvements and bug fixes.

Version 1.25 was released in Nov 2019. I would suggest following the debian docs own endorsement of installing newer versions as documented by Docker docs for Debian. You are already 3 years behind in development there.

You are coming a bit strong on me with your comment :) I only pointed out that a version of docker compose from 2019 still can run DMS just fine if version: 3 is added on top. This is a fact; I am aware that the version from universe is not officially endorsed by Debian nor up-to-date, but it is readily available when using Debian LTS and DMS works with it too.

It is possible that there are other docker compose quirks or features that make for a worse experience when using this old compose version, hence my original comment about documenting what is supported (and what not) in terms of Docker versions. My mistake was assuming that this package was part of Debian LTS packages (didn't check the repository), but it does not change the fact that it's better to explicitly mention which versions of Docker/compose are supported. Note that I made this comment with the intent of improving docs, so the defensive tone of your comment is out of my place from my point of view (as this was not an attack).

[polarathene]

You are coming a bit strong on me with your comment :)

Not intentional sorry 😅

but it is readily available when using Debian LTS and DMS works with it too.

There is no Debian LTS? Where do you get LTS from?

DMS may work, but we won't change docs to communicate that as we don't support such an old version for bug reports. If users want to deploy DMS with 3 year old version of docker compose, then they need to be aware of differences from modern config.

Soon the docker-compose.yml will change to compose.yaml, we can't take responsibility to maintain all the changes to accommodate those not willing to update, hopoe you can understand that 😅

---

It is possible that there are other docker compose quirks or features that make for a worse experience when using this old compose version, hence #2254 (comment) about documenting what is supported (and what not) in terms of Docker versions.

As just mentioned, that is not feasible for us, similar to older releases of DMS. There is only a few maintainers that volunteer time for support, none are able to offer support like that.

I know for certain that IPv6 support improved a fair bit since. And another bug with process soft / hard limit settings is being resolved in upcoming release of Docker Engine (waiting on fix from Golang). Building our image also requires BuildKit, preferably a release from at least 2022. The compose config that has become versionless supports a blend of V2 and V3 compose schemas now too. You can also look at changelogs for Docker Engine and Docker Compose to see all the changes over 3 years. Some also affects DNS within containers depending on host config.

---

My mistake was assuming that this package was part of Debian LTS packages (didn't check the repository), but it does not change the fact that it's better to explicitly mention which versions of Docker/compose are supported.

I assume by LTS packages, you just mean packages for that release of Debian? How does the upcoming Bookworm / 12 release in June affect you?

The docs used to have example configs with version: 3.8 if that helps. Other maintainers pushed for dropping the version to follow upstream Docker advice. Our docs will in the near future be updated again to align with Dockers docs in July focusing only on Docker Compose V2 (rewrite, not V2 schema). I don't think you'll see any support for earlier versions officially, you can still use that if it works for you, or share that information in bug reports, but if it can't be reproduced with V2 config maintainers may not be able to assist.

---

Note that I made this comment with the intent of improving docs, so the defensive tone of your comment is out of my place from my point of view (as this was not an attack).

I don't mean to come across as defensive / strong, sorry. Not the best communicator, my focus is on solving technical problems that arise in the project.

If old versions of software work well for you that is good, but our advice is to follow the official instructions from Dockers own docs for each distribution on adding their repos for much more recent releases.

[gdm85]

There is no Debian LTS? Where do you get LTS from?

https://wiki.debian.org/LTS

I assume by LTS packages, you just mean packages for that release of Debian? How does the upcoming Bookworm / 12 release in June affect you?

It does not: when using LTS versions one jumps from LTS to LTS. For example if you are on LTS 10 then once LTS 10 is no more supported you can switch to the next LTS available (11 or 12). Bookworm is currently not a candidate for LTS.

If old versions of software work well for you that is good, but our advice is to follow the official instructions from Dockers own docs for each distribution on adding their repos for much more recent releases.

It seems to me that your understanding of my use-case is just "person using old versions of software", however it is not: using old versions and never updating is generally a bad idea; using stable versions with security patches (that is what LTS is about) is a good idea instead, and a strategy to limit security issues (running the latest and greatest software also means running with the latest and greatest security issues).

So in fact it is dangerous (in general) to invite users to run the latest version of any software, because they get exposed to more security issues; instead the average user would benefit from a stable version with security patches, since they do not have a team of security experts checking what they're doing, all the time.

However I digress :) just wanted to explain the use-case. I am aware that most people run the latest version of everything and the security strategy is YOLO.

By the way, I noticed that the repository for dovecot was changed to "community" recently, that also made me think twice about the security aspect because security issues will hit that first and be unmitigated for all users this way.

As just mentioned, that is not feasible for us, similar to older releases of DMS. There is only a few maintainers that volunteer time for support, none are able to offer support like that.

Misunderstanding: I am not expecting that maintainers and volunteers test and document all possible combinations of docker compose used, I am not expecting anything but it would be nice if somewhere it is mentioned (for the dependencies, like docker, and plugins like compose or buildx) which minimal versions are necessary. This will save time to other people attempting to run DMS on different host configurations, and it is a good practice anyways because it basically means "at least this combination of versions is tested and reliable" (because the maintainers used it). Having a minimal supported version also helps with reproducibility of bugs/observations (think about system integration tests), and can be updated as new versions of the dependencies get released.

Expecting latest version of everything is not very future-proof, since software changes all the time; right now the documentation simply links to Docker's own documentation, so the message is "use the latest, whatever you find there".

I don't mean to come across as defensive / strong, sorry. Not the best communicator, my focus is on solving technical problems that arise in the project.

No problem :) thanks for your message, as you can see from above I also have purely technical reasons for writing things down; I did not make a PR for the docs to document minimal versions because I didn't know if you would want them there (seems like not) and I don't know what are such versions. But in conclusion if you prefer to leave things as they are no problem as well, my intention was to possibly make the installation process easier for other users installing DMS around.

[polarathene]

There is no Debian LTS? Where do you get LTS from?

https://wiki.debian.org/LTS

Thank you :)

using stable versions with security patches (that is what LTS is about) is a good idea instead, and a strategy to limit security issues (running the latest and greatest software also means running with the latest and greatest security issues).

So in fact it is dangerous (in general) to invite users to run the latest version of any software, because they get exposed to more security issues; instead the average user would benefit from a stable version with security patches

This honestly varies.

Your compose package for example, 1.25 from 2019 release, where are your security patches for it since? There has been many bug fixes and security patches since then. The package link doesn't seem to indicate much backporting from Debian going on there?

Docker backports fixes but only so far back. A recent issue that was addressed was with Go 1.19.9 and Go 1.20.4 releases that fixed how fd limits are configured implicitly by Go. Docker still needs to update their systemd service file, along with containerd, and a bug that could significantly slow down software or cause OOM will be resolved.

DMS with certain features enabled is affected by the slow down, although Debian isn't really affected because of a decision they have to workaround a bug with PAM (one that they introduced and haven't resolved last I checked).

That said, I am aware of the current Debian LTS Buster having more security fixes than Bullseye, as our Github Security tab shows with reported CVEs that affect Bullseye, but not Buster. I do understand the value of that, but tend to disagree the older the packages get.

For example CentOS 7 is still supported by Docker, it has a very old systemd version, it's not possible for users to upgrade to a newer one either unless they also update the kernel and perhaps some other parts of the system IIRC. Yet that affects decisions for resolving some issues in the current Docker development apparently (It was brought up as a concern when I proposed some changes), probably because despite the age, it's easier to upgrade Docker releases than it is to migrate to a newer release of CentOS 🤷‍♂️

By the way, I noticed that the repository for dovecot was changed to "community" recently, that also made me think twice about the security aspect because security issues will hit that first and be unmitigated for all users this way.

AFAIK, it was required to gain support for a feature that improved security.

Likewise it probably does include additional fixes and security patches that may not be backported to Dovecot package, if the compose package was anything to go by (which you yourself chose over the older LTS Buster package, despite your reasoning to prefer Buster and avoid Dockers official package repos).

I am not expecting anything but it would be nice if somewhere it is mentioned (for the dependencies, like docker, and plugins like compose or buildx) which minimal versions are necessary.

This will save time to other people attempting to run DMS on different host configurations, and it is a good practice anyways because it basically means "at least this combination of versions is tested and reliable" (because the maintainers used it).

Maintainers AFAIK follow Docker releases fairly closely, one might not be on 2023 releases yet. Our docs align with Dockers such as for the V2 Compose config / syntax. Older versions may work, but it's not something any maintainer can directly support. This is also less of a concern to us since it's rarely an issue to use Dockers repos for newer packages.

Having a minimal supported version also helps with reproducibility of bugs/observations (think about system integration tests), and can be updated as new versions of the dependencies get released.

I understand, but for us that's effectively our Github CI and test suite. It tends to work against our benefit to expand support further back, or to wider configurations.

At one point we had a macOS maintainer, they introduced changes to improve support there, but that ended up being dropped. They also added some other changes to fix / support their own environment with remote storage, but I'm not sure that was necessary / correct and likely ended up being a bit of a regression that's no longer simple to revert 😅

Usually our first go to if we can't reproduce a bug, is to request the reporter uses a more recent version of Docker software, as sometimes their release is just too old and misses fixes (especially for networking).

Expecting latest version of everything is not very future-proof, since software changes all the time; right now the documentation simply links to Docker's own documentation, so the message is "use the latest, whatever you find there".

Well that's not wrong. That's the extent of what we can support. If it works for you on older releases that's great, and we can still try to resolve issues that get reported, but if a newer version is the effective fix, then that's what we tend to advise.

Likewise for macOS and Windows users, or Podman and Kubernetes, support is quite limited on those fronts, releases of Docker from 2019 definitely falls into that same bucket 😅

I think v23 (Jan 2023) was a big update, and can serve as a good baseline for us. However I am still pushing towards getting docker.service and containerd.service changes upstreamed and would prefer that. There's some iptables rules fixes I'd also like to see taken care of.

my intention was to possibly make the installation process easier for other users installing DMS around.

It's rarely an issue in our experience, it's usually only been with the version field being dropped from our docs I think, and that affecting users with packages of Docker Compose before that was valid to do.

I did not make a PR for the docs to document minimal versions because I didn't know if you would want them there (seems like not) and I don't know what are such versions.

I don't think this is something the other maintainers would care much for, and usually that results in someone later removing it even if it did make it into the docs.

I'm not opposed to documenting incompatibilities, you can open a PR that references what I documented in this comment, 1.27 (Aug 2020) removed the version requirement, last release of 1.x series was May 2021 with deprecation notice in April 2022 and EOL in 2023.

Since we are more aligned with Docker as a docker focused project, I think it's fair for us to align with that EOL. Software may still work fine, it is just not encouraged and support will be lacking.

I completely understand if that's not ideal. I believe there is an alternative competitor on Github to DMS which is better backed financially that might meet the needs of users with different environment requirements/preferences :)