Forwarding (relaying) a specific domain over LMTP while keeping all others untouched (for mailman)

[Vosjedev]

Hello!

I'm trying to set up a mailman instance. Mailman has a bunch of options one should add to their postfix main.cf (see here):

transport_maps =
    hash:/path-to-mailman/var/data/postfix_lmtp
local_recipient_maps =
    hash:/path-to-mailman/var/data/postfix_lmtp
relay_domains =
    hash:/path-to-mailman/var/data/postfix_domains

however, scared that would mess with docker-mailserver, I'm trying to find an alternative way to do this. Mailman exposes an LMTP tcp socket (which I suppose these transport maps will point to).

Now the situation is this: I receive email on my domain, which is just my own email addresses. I configured a subdomain for my mailinglists, (atm it's an alias to an email account, I'm migrating from my own software that used IMAP to mailman), and would like postfix to use that for mailman. I'm guessing I can somehow make postfix aware it should forward email for that specific domain to mailman's LMTP, though I'm not sure if I can do that - I'm guessing there's a reason the official docs do it like this.

Anyone ideas on how I can approach this? My other idea was running a separate postfix container to forward the mail to, but that seems a bit overkill to me (and may cause other issues I'm not aware of, possibly).

[polarathene]

I can try assist with my knowledge of DMS internals, but I don't have much time to go over third-party software and verify anything 😓

1. How is the mail arriving into DMS? Over port 25?
2. You then want this mail to be delivered over LMTP to the mailman service? Based on the recipient address that the mail is arriving to DMS for?

When mail arrives on port 25 into Postfix, security checks are performed against the connecting client, which involves some checks like SPF validation, rDNS (not strict), etc. If all that passes the mail gets accepted and we decide what to do with it via Postfix config for recipients.

Postfix is configured for Dovecot mailbox accounts for delivery (via LMTP internally between the two services in the container), this is configured here:

docker-mailserver/target/scripts/startup/setup.d/postfix.sh

Lines 71 to 85 in cb76075

	# Dovecot feature integration
	# TODO: Alias SMTP_ONLY=0 to DOVECOT_ENABLED=1?
	if [[ ${SMTP_ONLY} -ne 1 ]]; then
	__postfix__log 'trace' 'Configuring Postfix with Dovecot integration'
	# /etc/postfix/vmailbox is created by: scripts/helpers/accounts.sh:_create_accounts()
	# This file config is for Postfix to verify a mail account exists before accepting
	# mail arriving and delivering it to Dovecot over LMTP.
	if [[ ${ACCOUNT_PROVISIONER} == 'FILE' ]]; then
	postconf 'virtual_mailbox_maps = texthash:/etc/postfix/vmailbox'
	fi
	# Historical context regarding decision to use LMTP instead of LDA (do not change this):
	# https://github.com/docker-mailserver/docker-mailserver/issues/4178#issuecomment-2375489302
	postconf 'virtual_transport = lmtp:unix:/var/run/dovecot/lmtp'
	fi

If you don't need the Dovecot mailbox and accounts integration you can opt-out via SMTP_ONLY=1, but you'd need to configure Postfix yourself for any mailbox domains/accounts it should accept mail for.

FWIW Postfix also uses /etc/passwd for system users which are associated to the DMS FQDN in the Postfix setting mydestination. I believe local_recipient_maps relates to these recipients, which isn't what you'd likely want to configure in DMS.

So the /etc/passwd internal system service accounts is via the local Postfix delivery agent, and would just dump the mail at /var/mail/<account name> instead of a Dovecot managed mailbox in /var/mail/<domain-part>/<local-part>/ via the virtual delivery agent configured for Dovecot LMTP.

relay_domains isn't something DMS presently has any official support for, that is just to say "hey accept mail for this recipient domain, and permit relaying it", which normally you can't do that if the sender address of the mail is not authorized to be sent from DMS (eg DNS records like SPF forbid it), although with SRS or ARC this can be successful, or when the receiving MTA that the mail is forwarded to is one you control to disregard the sender address.

If you instead submitted mail on port 587/465, the default permits you to send mail with any sender address (even if the receiving MTA would reject it), relay_domains is just to give permission for port 25 for select recipient domains, where anyone can send mail through DMS to that recipient (if there were no recipient domain restriction, then it'd make DMS an Open Relay which would be bad).

relay_domains will go through the relay transport (as per the docs for transport_maps). In DMS that is configured here (this configures the relay transport to use StartTLS, implicit TLS would require additional modification which I have shared config examples in past issues):

docker-mailserver/target/postfix/master.cf

Line 67 in cb76075

relay unix - - n - - smtp

We also have a related feature for relaying mail via ports 587/465 through a relay host (source, docs) which allows you to configure mail to be sent to the recipient from a separate MTA (like SendGrid or AWS SES) with credentials for those, and you can configure that by the sender address.

transport_maps takes priority at this stage above the relay host one. This way you can switch from the default relay transport to one you'd like to use instead (such as sending through LMTP to your mailman service).

The config syntax is documented here, where you provide a pattern to match on for the recipient address (it can be just local-part, domain-part, regex, full address, etc), and then what transport to map that to instead such as LMTP.

Here we can create a postfix-main.cf (DMS override config for Postfix settings) to add your desired transport_maps setting (DMS doesn't touch this setting presently so this shouldn't cause any issues):

transport_maps = texthash:/tmp/docker-mailserver/customizations/mailmain-lmtp

NOTE: texthash is used here instead of hash as it's just a tad simpler, no extra step with postmap involved to ensure the table is configured properly, just a plain text config file to use.

That file ./customizations/mailmain-lmtp lives in your DMS config volume with contents of one of these lines:

# Via Unix socket (file):
user@example.com    lmtp:unix:/var/run/mailman/lmtp

# Via container IP + port
user@example.com    lmtp:172.16.0.42:1337

If you have mailmain running on the host system or a separate container, you share the unix socket file via a volume mount into DMS at say /var/run/mailman/lmtp, and then the config example above would work. Or if it the service has an IP (but not 127.0.0.1) and you know what port the LMTP service is listening on, you can likewise connect via that way.

You can add more lines if you have multiple addresses to send 👍

---

The local_recipient_maps and relay_domains settings could be added to postfix-main.cf if needed, but I think all that is required is for DMS (Postfix rather) to acknowledge that it manages that mail domain. If you have any other mail accounts added you'd find /etc/vhosts would have the mail domains DMS considers it's own to manage.

relay_domains says it will look at relay_recipient_maps rather than local_recipient_maps to decide valid recipients for relaying. For LMTP delivery I believe you'd just need Postfix to recognize any valid recipient it can accept mail for (this also appears to be entirely optional but avoids spammers abusing backscatter by ensuring only valid recipients are accepted).

I'm pretty sure you can just use our setup email add ... command to create a mailbox in Dovecot, and due to transport_maps config, instead of mail being sent to Dovecot over the virtual transport, it'll be switched to your mailman service 😎

[Vosjedev]

I can try assist with my knowledge of DMS internals, but I don't have much time to go over third-party software and verify anything

of course, I didn't expect anyone to do so.

To answer the two points you noted:

1. How is the mail arriving into DMS? Over port 25?

Yes, regular SMTP over port 25.

2. You then want this mail to be delivered over LMTP to the mailman service? Based on the recipient address that the mail is arriving to DMS for?

Indeed. If I remember correctly, mailman wants to have postfix before it as a bouncer for unknown mail addresses (it lets postfix know which emailaddresses and domains are valid via these three automtically generated files). I'm going to do some tests with mailman to see how it responds to receiving email for mailaddresses it doesn't know about on it's LMTP socket though. My theory is that mailman doesn't send Your email could not be delivered emails, and that that would be the reason why they wanted postfix in front - I'm not fully sure yet, though.

I'm pretty sure you can just use our setup email add ... command to create a mailbox in Dovecot, and due to transport_maps config, instead of mail being sent to Dovecot over the virtual transport, it'll be switched to your mailman service 😎

The interesting thing is that these mailinglists will be automatically created in mailman via a third program, so updating the transport map and making DMS aware within a reasonable amount of time (can be a few hours given the lists won't receive a lot of usage).

all that is required is for DMS (Postfix rather) to acknowledge that it manages that mail domain

Which I guess I could be doing by creating one account under that domain? I need an account under the domain for sending from my third service anyways, so that would be no problem.

I think I'll be able to contruct a viable solution with this - either directly via config or via a script. Thank you very much for your in-depth answer (and sorry for my delay in responding, I've gotten quite busy this week), it cleared up a lot. I will update (and mark your answer as solution) when I've continued setting this up again (probably upcoming thursday) and have it working.

[polarathene]

The interesting thing is that these mailinglists will be automatically created in mailman via a third program, so updating the transport map and making DMS aware within a reasonable amount of time (can be a few hours given the lists won't receive a lot of usage).

The transport map config can be updated, just be mindful that if you bind mount this as a single file into the DMS container, if the inode of the file on the host were to change it would not sync within the container. That caveat is avoided by bind mounting a directory with the file instead, as the directory on the host doesn't have it's inode changed, any change to files contained in it get propagated. This isn't a docker or container specific quirk but is related to how the mount command works in linux IIRC.

Easiest way though is to just include the config in the same DMS config volume (/tmp/docker-mailserver) and I think that will work fine with texthash: + absolute filepath. You might need to verify if Postfix recognizes the change without running postfix reload (I know the equivalent for Dovecot is required for Dovecot config updates), otherwise just run that command and Postfix will be pretty quick to recognize the updated config (postfix reload has minimal down time of the service).

---

all that is required is for DMS (Postfix rather) to acknowledge that it manages that mail domain

Which I guess I could be doing by creating one account under that domain? I need an account under the domain for sending from my third service anyways, so that would be no problem.

Yes that should work. setup email add ... / setup alias add ... should both cause an update to /etc/postfix/vhost if there's anything new to add there.