security: postfix SMTP smuggling

[becker-s]

Subject

Something else that requires developers attention

Description

A vulnerability affecting several mail server implementations, including postfix, has been disclosed recently.
See the related postfix documentation page for more details: https://www.postfix.org/smtp-smuggling.html

While a short-term workaround exists, it is only partial:

NOTE: This will stop only the published form of the attack. Other forms exist that will not be stopped in this manner.

Postfix should be updated to a version in which the vulnerability is fixed (3.8.4, 3.7.9, 3.6.13 or 3.5.23) and the new optional feature smtpd_forbid_bare_newline be set to yes. The feature will be enabled by default for postfix >= 3.9.

If the smtpd_forbid_bare_newline feature cannot be enabled for backward-compatibility reasons, you could release a patch version of DMS with one of the patched postfix versions and let users decide whether they want to enable the feature via their local configuration.

[georglauterbach]

Current state of DMS' Postfix packaging (see https://packages.debian.org/de/postfix):

• :edge currently packs Debian 11, hence the version of Postfix is 3.5.18-0+deb11u1 and the vulnerability exists
• the new 13.1.0 release does not pack general: update base image to Debian 12 ("Bookworm") #3403 and hence contains the same version (3.5.18-0+deb11u1) as the current :edge
• when general: update base image to Debian 12 ("Bookworm") #3403 is merged, we will release v14.0.0, and Postfix will be updated to 3.7.6-0+deb12u2 where the vulnerability can be fixed? I am not sure how to properly read the website where the attack is described (https://www.postfix.org/smtp-smuggling.html): "A long-term fix is now available for Postfix 3.8.4, 3.7.9, 3.6.13 and 3.5.23." Is 3.7.6 included here?

Due to the way we organize releases, we cannot (or rather, it'd be immensely complicated to) backport such fixes. I do not know how we could provide users of DMS with versions <v14.0.0 with fixes.

CC @casperklein @polarathene

[georglauterbach]

Small note: @becker-s thank you very much for reporting this! :)

Next time, please use the dedicated security vulnerability reporting feature that we have enabled (https://github.com/docker-mailserver/docker-mailserver/security/advisories/new).

[becker-s]

Small note: @becker-s thank you very much for reporting this! :)

Next time, please use the dedicated security vulnerability reporting feature that we have enabled (https://github.com/docker-mailserver/docker-mailserver/security/advisories/new).

Sure thing! I seem to have missed the "Affected products" section when checking the security vulnerability issue type this morning, leading me to think that that type of issue is only meant for direct vulnerabilities rather than vulnerabilities in dependencies.

[becker-s]

Current state of DMS' Postfix packaging (see https://packages.debian.org/de/postfix):

* `:edge` currently packs Debian 11, hence the version of Postfix is `3.5.18-0+deb11u1` and the vulnerability exists

* the new 13.1.0 release does not pack [general: update base image to Debian 12 ("Bookworm") #3403](https://github.com/docker-mailserver/docker-mailserver/pull/3403) and hence contains the same version (`3.5.18-0+deb11u1`) as the current `:edge`

* when [general: update base image to Debian 12 ("Bookworm") #3403](https://github.com/docker-mailserver/docker-mailserver/pull/3403) is merged, we will release v14.0.0, and Postfix will be updated to `3.7.6-0+deb12u2` where the vulnerability can be fixed? I am not sure how to properly read the website where the attack is described (https://www.postfix.org/smtp-smuggling.html): "A long-term fix is now available for Postfix 3.8.4, 3.7.9, 3.6.13 and 3.5.23." Is 3.7.6 included here?

Due to the way we organize releases, we cannot (or rather, it'd be immensely complicated to) backport such fixes. I do not know how we could provide users of DMS with versions <v14.0.0 with fixes.

CC @casperklein @polarathene

According to the tracker page for the postfix Debian packages, only the version in unstable (3.8.4) has been released recently and contains the long-term fix. I assume that the Debian package maintainers will release the fixed version for Debian 12 (Bookworm) and then afterwards follow-up with a backport for Debian 11 (Buster) within the next days.

Unfortunately, the security researchers that discovered the vulnerability seem not to have included the postfix maintainers in their responsive disclosure process where proprietary mailing providers were informed. As such, both the postfix maintainers and the Debian package maintainers did not have to time to prepare and test versions where the vulnerability has been fixed.

EDIT: See the respective Debian security tracker entry for the status of the vulnerability in different Debian versions: https://security-tracker.debian.org/tracker/CVE-2023-51764

[casperklein]

Quick summary:

• Mitigation for all postfix versions for the published attack: smtpd_data_restrictions = reject_unauth_pipelining
  • Postfix 3.8.1, 3.7.6, 3.6.10 and 3.5.20 have a dedicated config option for that: smtpd_forbid_unauth_pipelining
• Postfix 3.8.4, 3.7.9, 3.6.13 and 3.5.23 provides additional mitigation for all attacks (including those not published yet) with the config option: smtpd_forbid_bare_newline`

For now, smtpd_data_restrictions = reject_unauth_pipelining should be set. But:

this may break legitimate SMTP clients that mis-implement SMTP, but such clients are exceedingly rare, especially when email is sent across the Internet.

I could imagine, Debian will release postfix updates for Debian 11/12 shortly.

[polarathene]

leading me to think that that type of issue is only meant for direct vulnerabilities rather than vulnerabilities in dependencies.

That's how I see it too.

If the vulnerability is already public knowledge, I don't see any issue with opening an issue here. At best you'd be minimizing awareness? 🤷‍♂️ I'd expect any capable malicious actor would have no issue finding the same public disclosure.

I do not know how we could provide users of DMS with versions <v14.0.0 with fixes.

Technically the scheduled build workflow could be adapted with manual dispatch with tag as an input. I think that would allow for building from the git tagged commit and publishing that tag to DockerHub and GHCR.

That should build with the package install process updating to the latest packages available, since we don't introduce any actual change beyond that it should only be packages with updated point releases (bug / security fixes). ClamAV database would be updated too.

Probably need to build without any cache used, which I don't think we presently support 🤔

[casperklein]

fyi: There will be a talk at 37C3 about SMTP smuggling.

[bjoerns1983]

Could you please give me an info how to properly insert the short term fix into postfix main.cf?
Can I simply include:

smtpd_data_restrictions = reject_unauth_pipelining
smtpd_discard_ehlo_keywords = chunking

into the mailserver.env, restart the container and it will be picked up by the postfix running inside the container?

[georglauterbach]

You need https://docker-mailserver.github.io/docker-mailserver/latest/config/advanced/override-defaults/postfix/ and add the lines there.

[bjoerns1983]

Really thanks for that hint.

Okay if i see this correct smtp_data_restrictions is already set to reject_unauth_pipelining in master.cf, so i only hat to change smtpd_discard_ehlo_keywords from silent-discard, dsn to chunking to implement the workaround. Is that correct?

[georglauterbach]

Really thanks for that hint.

Okay if i see this correct smtp_data_restrictions is already set to reject_unauth_pipelining in master.cf, so i only hat to changte smtpd_discard_ehlo_keywords from silent-discard, dsn to chunking to implement the workaround. Is taht correct?

smtp_data_restrictions only seems to be set for Amavis; setting it in main.cf should be done. I think that smtpd_discard_ehlo_keywords does not have to do with this problem. Once Postfix has an appropriate version, we can enable smtpd_forbid_bare_newline.

[bjoerns1983]

smtpd_discard_ehlo_keywords is at least mentioned in the postfix blogpost regarding the short term workaround.
Of course having a patched postfix version is the better way.

[polarathene]

smtpd_discard_ehlo_keywords is at east mentioned in the postfix blogpost regarding the short term workaround.

Just for reference, since this issue was opened the Postfix announcement link has since updated it's content, and includes a new "all versions" fix:

smtpd_data_restrictions = reject_unauth_pipelining
smtpd_discard_ehlo_keywords = chunking

smtpd_discard_ehlo_keywords = chunking was also added to the original short-term fix.

Previously neither the short-term or long-term fixes were viable for DMS as the short-term fix smtpd_forbid_unauth_pipelining = reject_unauth_pipelining requires a newer Postfix version than Debian packages provide (Bookworm provides the necessary 3.7.6 which DMS v14 will have, but Bullseye still has 3.5.18).

[woeisme]

smtpd_discard_ehlo_keywords is at east mentioned in the postfix blogpost regarding the short term workaround.

Just for reference, since this issue was opened the Postfix announcement link has since updated it's content, and includes a new "all versions" fix:

smtpd_data_restrictions = reject_unauth_pipelining
smtpd_discard_ehlo_keywords = chunking

smtpd_discard_ehlo_keywords = chunking was also added to the original short-term fix.

Previously neither the short-term or long-term fixes were viable for DMS as the short-term fix smtpd_forbid_unauth_pipelining = reject_unauth_pipelining requires a newer Postfix version than Debian packages provide (Bookworm provides the necessary 3.7.6 which DMS v14 will have, but Bullseye still has 3.5.18).

For those of us coming here after the talk at 37c3, is the correct procedure to 'fix' this problem to:

sudo nano docker-data/dms/config/postfix-main.cf

then pasting the lines:

smtpd_data_restrictions = reject_unauth_pipelining smtpd_discard_ehlo_keywords = chunking
and finally restarting the docker instance (in my case):

docker-compose stop
followed by
docker-compose start

(btw the documenation update instructions incorrectly say 'docker compose' instead of 'docker-compose')

It would be nice to get a followup comment from someone else here confirming this is the accepted procedure for patching this bug.

[georglauterbach]

docker-compose stop followed by docker-compose start

Please don't! (see https://docker-mailserver.github.io/docker-mailserver/latest/config/debugging/#preliminary-checks)

(btw the documenation update instructions incorrectly say 'docker compose' instead of 'docker-compose')

The docs are correct; you seem to be using an older version of Docker Compose. The latest versions are a plugin to the docker command, hence docker compose is correct (and up to date) (ref: https://docs.docker.com/compose/reference/).

It would be nice to get a followup comment from someone else here confirming this is the accepted procedure for patching this bug.

CC @polarathene | What do we do with smtpd_discard_ehlo_keywords where silent-discard, dsn seem to be set?

[polarathene]

What do we do with smtpd_discard_ehlo_keywords where silent-discard, dsn seem to be set?

This was provided via PR review feedback. chunking can be prepended/appended, but this is only necessary for the short-term fixes. Unless there is a good reason to keep that addition, it'll be reverted once the long-term fix can be configured.

---

It would be nice to get a followup comment from someone else here confirming this is the accepted procedure for patching this bug.

• Add a postfix-main.cf config override if one doesn't yet exist.

• Add the following to postfix-main.cf:

  # SMTP smuggling short-term fix (DMS v14 will make these two lines redundant):
  smtpd_data_restrictions = reject_unauth_pipelining
  smtpd_discard_ehlo_keywords = chunking
  

  NOTE: That will remove a recent addition to drop support for DSNs on inbound port 25 (while submission ports 465 + 587 allow DSNs for authenticated users). If you want to keep that, you want to modify that line to smtpd_discard_ehlo_keywords = chunking, silent-discard, dsn

• docker compose down && docker compose up (or similar to correctly restart DMS).

• When DMS v14 is out, remove those lines 👍

• Future release after v14 will switch to the long-term fix once Debian makes the updated packages available.

[bjoerns1983]

Debians Bullseye Updated Repo has been updated to Postfix 3.5.23 which should contain the longterm fix.
https://packages.debian.org/search?suite=bullseye-updates&searchon=names&keywords=postfix

[georglauterbach]

I will trigger a build so :edge has the appropriate version. A built-in fix in DMS should be around in the next few days I think.

[georglauterbach]

I will re-open; because we should update when merged Debian 12 (right?).

[polarathene]

I will re-open; because we should update when merged Debian 12 (right?).

Do we have an actual blocker related to that reason? I think we resolved this with the v13.2 release right? (which you released a day after the above comment)

docker-mailserver/CHANGELOG.md

Lines 45 to 51 in 2bf5234

	## [v13.2.0](https://github.com/docker-mailserver/docker-mailserver/releases/tag/v13.2.0)
	### Security
	DMS is now secured against the [recently published spoofing attack "SMTP Smuggling"](https://www.postfix.org/smtp-smuggling.html) that affected Postfix ([#3727](https://github.com/docker-mailserver/docker-mailserver/pull/3727)):
	- Postfix upgraded from `3.5.18` to `3.5.23` which provides the [long-term fix with `smtpd_forbid_bare_newline = yes`](https://www.postfix.org/smtp-smuggling.html#long)

[georglauterbach]

I thought there was a change we could only implement with Debian 12; but I guess I was wrong. I'll close this then.

[polarathene]

I thought there was a change we could only implement with Debian 12

The newer Bullseye Postfix patch release backported the long-term fix IIRC.