question: How to configure Traefik to proxy inbound mail connections via Port 25 (StartTLS)

[vincentDcmps]

Subject

I would like some feedback concerning a use case

Description

HI I try to configure docker mail server behind traefik following thins works:
send internal mail from outside
send mail to outside
note: for that I must add "traefik.tcp.routers.esmtp.tls.passthrough=true" to esmtp and imap on traefik

but I can't receive mail from another domain behind traefik but telnet on port 25 works
I have try to put docker-mail server port 25 in front and in this case everything works
I have correctly put following parameter (using edge)

submission/inet/smtpd_upstream_proxy_protocol=haproxy
submissions/inet/smtpd_upstream_proxy_protocol=haproxy

and

postscreen_upstream_proxy_protocol = haproxy

try to change traefik service terminaison time but doesn't work too

I don't see any log for this request in DMS so I suposse that traefik drop request but don't know why or missing something in postscreen config for reverse proxy?

[polarathene]

You've not filled a proper bug template. It's usually helpful not just for maintainers but others that arrive here with the same issue to have more information about your configuration.

Just to confirm, have you verified you can connect to Port 465 via Traefik? Two users are struggling with this currently with ThunderBird unable to connect via Traefik to submit mail, but successful if connecting to DMS port 465 directly:

• question: (Traefik + haproxy) - How to configure for DMS? #3547 (comment)
• question: How to configure Traefik for port 465? (Postfix smtpd throws TLS library error) #3546

---

HI I try to configure docker mail server behind traefik following thins works:
send internal mail from outside
send mail to outside
note: for that I must add "traefik.tcp.routers.esmtp.tls.passthrough=true" to esmtp and imap on traefik

ESMTP is an improvement on SMTP, but it's not a specific port. You refer to sending mail, and that you must enable TLS passthrough for it to work.

Are you referring to port 465 or 587 here? Or also outbound port 25?

• Ports 587 (StartTLS) + 465 (implicit TLS, this one needs passthrough) are specifically for mail submission (to be delivered internally, or sent outbound on port 25).
• Port 25 inbound is for receiving mail only, not intended for submitting mail to be sent.
• Port 25 outbound is used when sending mail to another mail server without authentication.
• Port 25 uses StartTLS, thus passthrough in Traefik isn't likely to be relevant since it's explicit. Must start as plain-text connection and be upgraded to TLS via negotiation of the StartTLS protocol.

Can you clarify the difference between:

• "send internal mail from outside"
• "send mail to outside"

Internal mail sent from outside? Is this referring to a mail client like ThunderBird successfully connecting to DMS to send mail to DMS mail account?

Send mail to outside. How are you testing this one? Is your mail client from the outside not able to also accomplish this? It should if it connects to DMS on port 587 / 465 as that authenticates with a DMS account to establish trust. Port 25 is not given this trust as there is no authentication (since the port is only intended for receiving inbound mail, and sending authorized outbound mail).

I can't receive mail from another domain behind traefik

By "another", do you mean you can receive mail for an account that DMS will accept? How does it differ from one that fails? Or are you referring to only mail with a sender address of a DMS account is accepted?

Configuration being shared here would be helpful. I'd like to know:

• Are you using a subdomain for DMS like mail.example.com, with accounts like hello@example.com? (NOTE: LDAP presently drops support for internal system users and mail being able to deliver mail, something that seems to have been mistakenly contributed a long time ago)
• How are you testing this mail delivery?
  • Is this a deployed server and you're sending mail from another external mail service to deliver to DMS?
  • Are you sending mail locally though to Traefik to route instead? (eg: Mail submission over port 465)

I don't see any log for this request in DMS so I suposse that traefik drop request but don't know why or missing something in postscreen config for reverse proxy?

See the last section of my response here. It should be helpful to debug against internal DMS container, and also against the Traefik port from host/container that should route to DMS.

[vincentDcmps]

Just to confirm, have you verified you can connect to Port 465 via Traefik? Two users are struggling with this currently with ThunderBird unable to connect via Traefik to submit mail, but successful if connecting to DMS port 465 directly:

yes port 465 and 993 works correctly just need to add "tls.passthrough=true" for each in traefik config

Are you referring to port 465 or 587 here? Or also outbound port 25?

esmtp is port 465 in my configuration
issue is only in port 25 inbound

   "send internal mail from outside"

mail I use on DMS in test@domain.eu I can send a mail to test2@domain.eu with success

"send mail to outside"

I can send a mail from test@domain.eu to test@gmail.com

I can't receive mail from another domain behind traefik

if I send mail from test@gmail.com to test@domain.eu i don't receive the mail

Are you using a subdomain for DMS like mail.example.com, with accounts like hello@example.com? (NOTE: LDAP presently drops support for internal system users and mail being able to deliver mail, something that seems to have been mistakenly contributed a long time ago)

yes I use a subdomain like mail.domain.eu

[polarathene]

if I send mail from test@gmail.com to test@domain.eu I don't receive the mail

Can you drop the Traefik reverse proxy for port 25, and verify that DMS will work correctly then?

[vincentDcmps]

Yes DMS works correctly on port 25 without traefik

[polarathene]

Yes DMS works correctly on port 25 without traefik

You should share more information, such as config then.

The two other Traefik issues AFAIK have Port 25 working with Traefik to receive mail, but unlike you have trouble with Port 465 🤷‍♂️

Something must be different between your configs or environments.

---

Both seem to have configured Port 25 like this (which seems to match an example from our docs):

  labels:
      - "traefik.enable=true"
      - "traefik.tcp.routers.smtp.rule=HostSNI(`*`)"
      - "traefik.tcp.routers.smtp.entrypoints=smtp"
      - "traefik.tcp.routers.smtp.service=smtp"
      - "traefik.tcp.services.smtp.loadbalancer.server.port=25"
      - "traefik.tcp.services.smtp.loadbalancer.proxyProtocol.version=1"

If I understand the Traefik docs right, this leaves TLS enabled to be terminated by Traefik (no passthrough), but this port like 587 should not be using implicit TLS.

See the following references:

• Unable to connect over TCP with STARTTLS traefik/traefik#5688 (comment)
• https://community.traefik.io/t/v2-tcp-router-with-tls-example/2664/2
• SMTP-STARTTLS proxying with STARTTLS handled by Traefik traefik/traefik#7366 (comment)
• Traefik 3.0: StartTLS connection hanging if connection initiated when upstream unavailable traefik/traefik#9929 (comment)

It seems you need to use plain TCP router, and have the connection negotiate StartTLS from client to DMS. Traefik can't leverage SNI in router without TLS, so must be wildcard.

[vincentDcmps]

I have pretty the same for port 25

        "traefik.enable=true",
        "traefik.tcp.routers.smtp.service=smtp",
        "traefik.tcp.routers.smtp.entrypoints=smtp",
        "traefik.tcp.routers.smtp.rule=HostSNI(`*`)",
        "traefik.tcp.services.smtp.loadbalancer.proxyProtocol.version=1",

doesn't use "traefik.tcp.services.smtp.loadbalancer.server.port=25" because on server side port is not 25 I use consul/nomad as service discovery/orchestrator

so port diagram is like this

flowchart LR
    A[outside] -->|port 25| B[traefik]
    B --> |random port| C[docker]
    C -->|port 25| D[postfix]

[vincentDcmps]

traefik config file

        [entryPoints]

          [entrypoints.ssh]
            address = ":2222"
          [entryPoints.web]
            address = ":80"
            [entryPoints.web.http]
              [entryPoints.web.http.redirections]
                [entryPoints.web.http.redirections.entryPoint]
                  to = "websecure"
                  scheme = "https"
          [entryPoints.websecure]
            address = ":443"
          [entryPoints.traefik]
            address = ":9080"
          [entrypoints.smtp]
            address = ":25"
          [entrypoints.esmtp]
            address = ":465"
          [entrypoints.imap]
            address = ":993"
        [http.middlewares]
          [http.middlewares.https-redirect.redirectscheme]
            scheme = "https"
        [providers.consulCatalog]
          exposedByDefault = false
          [providers.consulCatalog.endpoint]
            address = "{{{env "NOMAD_IP_admin"}}}:8500"
        [log]
        [accessLog]
        [api]
          dashboard = true
          insecure = true
        [ping]
        [certificatesResolvers.myresolver.acme]
        email = "vincent@ducamps.win"
        storage = "acme.json"
        [certificatesResolvers.myresolver.acme.httpChallenge]
        entryPoint= "web"
        [metrics]
          [metrics.prometheus]

[polarathene]

Did you follow advice in my last message to ensure Traefik port 25 router is not using TLS?

As mentioned here, Traefik doesn't yet support STARTTLS, so advice is to:

• Have a plain-text TCP router to DMS which will negotiate STARTTLS to upgrade the connection to be secure (DMS handles the certs, which it can import from traefik via acme.json if needed).
• Have the DMS routed service use haproxy protocol like you've configured already. This retains the real client IP to avoid any accidental open relay.

The same should work for port 587 or any other DMS port with STARTTLS. You only need to be concerned with Port 25 and can use implicit TLS everywhere else though.

[vincentDcmps]

yes I have try to set off tls on traefik router and same issue

[polarathene]

I am out of ideas then 😅

Perhaps your addition of consul / nomad is affecting this in some way since the other Traefik users don't seem to have this problem 🤷‍♂️

It would be great to know what is different between your environments that causes the inconsistency.

---

• You could try ensure that it's IPv4 traffic all the way through, no switches to IPv6.
  • Docker releases have also added quite a lot of networking fixes, ideally you have a release from 2023.
  • Some hosts have firewalls or other security layers active that may also influence networking behaviour, perhaps check your system logs if Traefik or consul/nomad and DMS don't seem to communicate any additional insights to where traffic is getting stuck.
• Maybe try userland-proxy: false setting in /etc/docker/daemon.json (requires restarting the docker service).
• I think it's unlikely, but depending on your host OS, you may hit an issue with containers having excessive soft limit for number of file descriptors.
  • In the DMS container you can run ulimit -Sn, ideally this is 1024, if it's around 1 million, that's not great but shouldn't be too impactful, if it's over a billion then it's possible some software may misbehave.
  • This can be fixed with systemctl edit docker.service + systemctl edit containerd.service usually. Your host ulimit -Sn should be fine, only an issue in containers when the service file configures LimitNOFILE=infinity. It should be LimitNOFILE=1024:524288. Releases from Docker and containerd later this year will have fixed this.

---

It might turn out that you're affected by this: traefik/traefik#9929

Although that seems to be about Traefik v3, and possibly specific to their Postgres STARTTLS support feature, but the discussion at the end seems to suggest it might be a bug with any starttls connection?

[vincentDcmps]

thanks for your help
traefik's log from TCP router are very poor.
for now I will keep DMS port 25 in front
and try to follow this trafiek issue to test again

By the way do we need to add the tls.passthrough=true for other port in DMS docs?

[polarathene]

By the way do we need to add the tls.passthrough=true for other port in DMS docs?

Probably yes to any port that is for implicit TLS:

[vincentDcmps]

Just found root cause of my proxy issue according traefik documentation https://doc.traefik.io/traefik/routing/routers/#entrypoints_1 "How to handle Server First protocols?"
I have got all my traefik router set to listen on all entry point I have change this to only listen on web entrypoint and now proxy worked on port 25 thanks for you support!