Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Virtualmin does not include DKIM/SPF/DMARC information when an email is generated via the command line #785

Open
c-prompt opened this issue Apr 16, 2024 · 23 comments

Comments

@c-prompt
Copy link

After 5 solid days of frustrating troubleshooting, I'm calling uncle.

I have 2 servers, both of which have issues assigning DKIM/SPF/DMARC information when an email is generated via the command line mail program. The Virtualmin domain is setup as "sub.myservername.com" with the hostname as "host." All emails are sent properly via Postfix (i.e., with valid DKIM/SPF/DMARC information) with normal email clients and they are also all sent correctly when I test using command line mail as "sub.myservername.com" instead of "host.sub.myservername.com". In other words, the following works properly:

echo "Test message - you know the drill" | mail -r "root <root@sub.myservername.com>" -s "Test Message" test-d1b79814@appmaildev.com

However, none of the DKIM/SPF/DMARC information is sent when an email is generated using command line mail (as sometimes occurs when an application sends an error report) which always seems to come from "root@host.sub.myservername.com" automatically. I haven't found a way to change the automatic email to come from, e.g., root@sub.myservername.com instead of root@host.sub.myservername.com. I even tried changing my /root/.mailrc to:

set name="Root Mail"
set from="root@sub.myservername.com"

...but it doesn't make a difference and the command line emails are still sent from root@host.sub.myservername.com.

The use case: when certain errors occur on my servers, I have the error reports sent both to my Virtualmin accounts as well as Gmail accounts (in case I can't get access to the emails on Virtualmin because of the problem which has happened in the past). However, Gmail is rejecting those emails because:

my_gmail_address@gmail.com (expanded from ): host
gmail-smtp-in.l.google.com[173.194.211.26] said: 550-5.7.26 This mail has
been blocked because the sender is unauthenticated. 550-5.7.26 Gmail
requires all senders to authenticate with either SPF or DKIM. 550-5.7.26
550-5.7.26 Authentication results: 550-5.7.26 DKIM = did not pass
550-5.7.26 SPF [host.sub.myservername.com] with ip:
[server_ip_address 550-5.7.26 ] = did not pass 550-5.7.26 550-5.7.26 For
instructions on setting up authentication, go to 550 5.7.26
https://support.google.com/mail/answer/81126#authentication
if7-20020a0561022c8700b0047b875f02d0si1005655vsb.248 - gsmtp (in reply to
end of DATA command)

In other words, Gmail is rejecting the emails because the command line script that sends the errors doesn't have a valid SPF or DKIM.

For example, using the excellent diagnostic tool at https://www.appmaildev.com/, I executed this at the command line:

echo "Test message - you know the drill" | mail -r "root <root@host.sub.myservername.com>" -s "Test Message" test-d1b79814@appmaildev.com

...which generated the following summary:

SPF: None
DKIM: permerror (no key)
DMARC: permerror

...and which generated the following detailed report:

============================================================================
This is SPF/DKIM/DMARC/RBL report generated by a test tool provided 
	by AdminSystem Software Limited.

Any problem, please contact support@emailarchitect.net
============================================================================
Report-Id: 46f06971
Sender: <root@host.sub.myservername.com>
Header-From: <root@host.sub.myservername.com>
HELO-Domain: host.sub.myservername.com
Source-IP: server_ip_address
SSL/TLS: TLS secured
Validator-Version: 1.19
============================================================================
Original email header:

x-sender: root@host.sub.myservername.com
x-receiver: test-46f06971@appmaildev.com
Received: from host.sub.myservername.com ([server_ip_address]) by appmaildev.com over TLS secured channel with Microsoft SMTPSVC(8.5.9600.16384);
	 Tue, 16 Apr 2024 21:35:13 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=host.sub.myservername.com; s=202404; t=1713303311;
	bh=e7R0pFzcLDW2cd86kMjeWyx8JMvgFlzFN7EZwVhpr+Y=;
	h=From:To:Subject:Date:From;
	b=IASyRJP06sfJxW730hfG/uttHq5+zSpx2oLW+RCdLj8B71bskEvYJWiVx3grEABrX
	 vnd5zFEcCn6x5saXcLRy1bBIimZQcU511a7iH4vR6k6YQTDP7aVnAplDG3MOf5XbFo
	 ZYBKm02mwAkkhx4eTsXsSrmKaqi35Bf7sQQ3jlCD04MIGhQIa0tUbwkZmTM1apnxgQ
	 qXjtLjpjuTwDykZLSNUVee4nIoOScXmDM5WDZZAL01yzanyGHtVkZ6ICtHj90Oq3YR
	 o17dfMojmUXsvO9xXFqXVzMdcoV5an2S9Em7u4yuMXfIgKd5zzJ8k8o3MVv5MO0LrA
	 FAIrEpybLU7yg==
Received: by host.sub.myservername.com (Postfix, from userid 0)
	id 809C3673C9; Tue, 16 Apr 2024 17:35:11 -0400 (EDT)
From: root <root@host.sub.myservername.com>
To: test-46f06971@appmaildev.com
Subject: Test Message
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Message-Id: <20240416213511.809C3673C9@host.sub.myservername.com>
Date: Tue, 16 Apr 2024 17:35:11 -0400 (EDT)
Return-Path: root@host.sub.myservername.com
X-OriginalArrivalTime: 16 Apr 2024 21:35:14.0036 (UTC) FILETIME=[F7801340:01DA9045]

============================================================================
SPF: None
============================================================================

Sender-IP: server_ip_address
Sender-Domain-Helo-Domain: host.sub.myservername.com

Query TEXT record from DNS server for: host.sub.myservername.com
Exception: No records found for given DNS query

============================================================================
DKIM: permerror
============================================================================

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=host.sub.myservername.com; s=202404; t=1713303311;
	bh=e7R0pFzcLDW2cd86kMjeWyx8JMvgFlzFN7EZwVhpr+Y=;
	h=From:To:Subject:Date:From;
	b=IASyRJP06sfJxW730hfG/uttHq5+zSpx2oLW+RCdLj8B71bskEvYJWiVx3grEABrX
	 vnd5zFEcCn6x5saXcLRy1bBIimZQcU511a7iH4vR6k6YQTDP7aVnAplDG3MOf5XbFo
	 ZYBKm02mwAkkhx4eTsXsSrmKaqi35Bf7sQQ3jlCD04MIGhQIa0tUbwkZmTM1apnxgQ
	 qXjtLjpjuTwDykZLSNUVee4nIoOScXmDM5WDZZAL01yzanyGHtVkZ6ICtHj90Oq3YR
	 o17dfMojmUXsvO9xXFqXVzMdcoV5an2S9Em7u4yuMXfIgKd5zzJ8k8o3MVv5MO0LrA
	 FAIrEpybLU7yg==
Signed-by: root@host.sub.myservername.com
Expected-Body-Hash: e7R0pFzcLDW2cd86kMjeWyx8JMvgFlzFN7EZwVhpr+Y=
Current Utc timestamp: 2024-04-16T21:35:16.489; Signature timestamp: 2024-04-16T21:35:11.000

Canonicalized header: from:root <root@host.sub.myservername.com>
to:test-46f06971@appmaildev.com
subject:Test Message
date:Tue, 16 Apr 2024 17:35:11 -0400 (EDT)


DKIM-Result: permerror (no key)

============================================================================
DMARC: permerror
============================================================================

_dmarc.myservername.com: v=DMARC1; p=none; pct=100
Received-SPF: none (appmaildev.com: domain of root@host.sub.myservername.com does not designates permitted sender) client-ip=server_ip_address
Authentication-Results: appmaildev.com;
    dkim=permerror (no key) header.d=host.sub.myservername.com;
    spf=none (appmaildev.com: domain of root@host.sub.myservername.com does not designates permitted sender) client-ip=server_ip_address;
    dmarc=permerror (adkim=r aspf=r p=none) header.from=host.sub.myservername.com;

============================================================================
DomainKey: none
============================================================================

DomainKey-Result: none (no signature)
If DKIM result is passed, you can ignore DomainKey result: none
Notice: DomainKey is obsoleted standard, the new standard is DKIM.

============================================================================
PTR: ExistsRecord
============================================================================

Sender-IP: server_ip_address
Query 189.66.45.38.in-addr.arpa
Host: host.sub.myservername.com

============================================================================
RBL: NotListed
============================================================================

bl.spamcop.net:Not Listed (OK) - http://bl.spamcop.net 
cbl.abuseat.org:Not Listed (OK) - http://cbl.abuseat.org 
b.barracudacentral.org:Not Listed (OK) - http://www.barracudacentral.org/rbl/removal-request 
dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
http.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
dul.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
misc.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
smtp.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
socks.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
spam.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
web.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
zombie.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
pbl.spamhaus.org:Not Listed (OK) - http://www.spamhaus.org/pbl/ 
sbl.spamhaus.org:Not Listed (OK) - http://www.spamhaus.org/sbl/ 
xbl.spamhaus.org:Not Listed (OK) - http://www.spamhaus.org/xbl/ 
zen.spamhaus.org:Not Listed (OK) - http://www.spamhaus.org/zen/ 
ubl.unsubscore.com:Not Listed (OK) - http://www.lashback.com/blacklist/ 
rbl.spamlab.com:Not Listed (OK) - http://tools.appriver.com/index.aspx?tool=rbl 
dyna.spamrats.com:Not Listed (OK) - http://www.spamrats.com 
noptr.spamrats.com:Not Listed (OK) - http://www.spamrats.com 
spam.spamrats.com:Not Listed (OK) - http://www.spamrats.com 
dnsbl.inps.de:Not Listed (OK) - http://dnsbl.inps.de/index.cgi?lang=en 
drone.abuse.ch:Not Listed (OK) - http://dnsbl.abuse.ch 
httpbl.abuse.ch:Not Listed (OK) - http://dnsbl.abuse.ch 
korea.services.net:Not Listed (OK) - http://korea.services.net 
short.rbl.jp:Not Listed (OK) - http://www.rbl.jp 
virus.rbl.jp:Not Listed (OK) - http://www.rbl.jp 
spamrbl.imp.ch:Not Listed (OK) - http://antispam.imp.ch 
wormrbl.imp.ch:Not Listed (OK) - http://antispam.imp.ch 
virbl.bit.nl:Not Listed (OK) - http://virbl.bit.nl  
rbl.suresupport.com:Not Listed (OK) - http://suresupport.com/postmaster 
dsn.rfc-ignorant.org:Not Listed (OK) - http://www.rfc-ignorant.org/policy-dsn.php 
spamguard.leadmon.net:Not Listed (OK) - http://www.leadmon.net/SpamGuard/ 
dnsbl.tornevall.org:Not Listed (OK) - http://opm.tornevall.org 
netblock.pedantic.org:Not Listed (OK) - http://pedantic.org 
multi.surbl.org:Not Listed (OK) - http://www.surbl.org 
ix.dnsbl.manitu.net:Not Listed (OK) - http://www.dnsbl.manitu.net 
tor.dan.me.uk:Not Listed (OK) - http://www.dan.me.uk/dnsbl 
rbl.efnetrbl.org:Not Listed (OK) - http://rbl.efnetrbl.org 
access.redhawk.org:Not Listed (OK) - http://www.redhawk.org/index.php?option=com_wrapper&Itemid=33 
db.wpbl.info:Not Listed (OK) - http://www.wpbl.info 
rbl.interserver.net:Not Listed (OK) - http://rbl.interserver.net 
query.senderbase.org:Not Listed (OK) - http://www.senderbase.org/about 
bogons.cymru.com:Not Listed (OK) - http://www.team-cymru.org/Services/Bogons/ 
csi.cloudmark.com:Not Listed (OK) - http://www.cloudmark.com/en/products/cloudmark-sender-intelligence/index 

cbl.anti-spam.org.cn:DnsTimeout - http://www.anti-spam.org.cn/?Locale=en_US 
cdl.anti-spam.org.cn:DnsTimeout - http://www.anti-spam.org.cn/?Locale=en_US 
dnsbl.dronebl.org:DnsTimeout - http://www.dronebl.org 


============================================================================
Original message source
============================================================================
x-sender: root@host.sub.myservername.com
x-receiver: test-46f06971@appmaildev.com
Received: from host.sub.myservername.com ([server_ip_address]) by appmaildev.com over TLS secured channel with Microsoft SMTPSVC(8.5.9600.16384);
	 Tue, 16 Apr 2024 21:35:13 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=host.sub.myservername.com; s=202404; t=1713303311;
	bh=e7R0pFzcLDW2cd86kMjeWyx8JMvgFlzFN7EZwVhpr+Y=;
	h=From:To:Subject:Date:From;
	b=IASyRJP06sfJxW730hfG/uttHq5+zSpx2oLW+RCdLj8B71bskEvYJWiVx3grEABrX
	 vnd5zFEcCn6x5saXcLRy1bBIimZQcU511a7iH4vR6k6YQTDP7aVnAplDG3MOf5XbFo
	 ZYBKm02mwAkkhx4eTsXsSrmKaqi35Bf7sQQ3jlCD04MIGhQIa0tUbwkZmTM1apnxgQ
	 qXjtLjpjuTwDykZLSNUVee4nIoOScXmDM5WDZZAL01yzanyGHtVkZ6ICtHj90Oq3YR
	 o17dfMojmUXsvO9xXFqXVzMdcoV5an2S9Em7u4yuMXfIgKd5zzJ8k8o3MVv5MO0LrA
	 FAIrEpybLU7yg==
Received: by host.sub.myservername.com (Postfix, from userid 0)
	id 809C3673C9; Tue, 16 Apr 2024 17:35:11 -0400 (EDT)
From: root <root@host.sub.myservername.com>
To: test-46f06971@appmaildev.com
Subject: Test Message
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Message-Id: <20240416213511.809C3673C9@host.sub.myservername.com>
Date: Tue, 16 Apr 2024 17:35:11 -0400 (EDT)
Return-Path: root@host.sub.myservername.com
X-OriginalArrivalTime: 16 Apr 2024 21:35:14.0036 (UTC) FILETIME=[F7801340:01DA9045]

Test message - you know the drill
============================================================================

When the email is sent, I can see in my syslog:

2024-04-16T17:07:12.515010-04:00 host opendkim[788]: 766B1673CB: DKIM-Signature field added (s=202404, d=host.sub.myservername.com)

My etc/postfix/main.cf:

# See /usr/share/postfix/main.cf.dist for a commented, more complete version

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 3.6 on
# fresh installs.
compatibility_level = 3.6

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level = may

smtp_tls_CApath=/etc/ssl/certs
smtp_tls_security_level = dane
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = host.sub.myservername.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, host.sub.myservername.com, localhost.sub.myservername.com, , localhost
relayhost = 
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
virtual_alias_maps = hash:/etc/postfix/virtual
sender_bcc_maps = hash:/etc/postfix/bcc
sender_dependent_default_transport_maps = hash:/etc/postfix/dependent
home_mailbox = Maildir/
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
smtp_dns_support_level = dnssec
smtp_host_lookup = dns
allow_percent_hack = no
resolve_dequoted_address = no
tls_server_sni_maps = hash:/etc/postfix/sni_map
milter_default_action = accept
smtpd_milters = inet:127.0.0.1:8891
non_smtpd_milters = inet:127.0.0.1:8891
disable_vrfy_command = yes

My etc/postfix/postfix master.cf:

#
# Postfix master process configuration file.  For details on the format
# of the file, see the master(5) manual page (command: "man 5 master" or
# on-line: http://www.postfix.org/master.5.html).
#
# Do not forget to execute "postfix reload" after editing this file.
#
# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (no)    (never) (100)
# ==========================================================================
smtp	inet	n	-	y	-	-	smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may
#smtp      inet  n       -       y       -       1       postscreen
#smtpd     pass  -       -       y       -       -       smtpd
#dnsblog   unix  -       -       y       -       0       dnsblog
#tlsproxy  unix  -       -       y       -       0       tlsproxy
# Choose one: enable submission for loopback clients only, or for any client.
#127.0.0.1:submission inet n -   y       -       -       smtpd
#submission inet n       -       y       -       -       smtpd
#  -o syslog_name=postfix/submission
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_tls_auth_only=yes
#  -o smtpd_reject_unlisted_recipient=no
#     Instead of specifying complex smtpd_<xxx>_restrictions here,
#     specify "smtpd_<xxx>_restrictions=$mua_<xxx>_restrictions"
#     here, and specify mua_<xxx>_restrictions in main.cf (where
#     "<xxx>" is "client", "helo", "sender", "relay", or "recipient").
#  -o smtpd_client_restrictions=
#  -o smtpd_helo_restrictions=
#  -o smtpd_sender_restrictions=
#  -o smtpd_relay_restrictions=
#  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
# Choose one: enable submissions for loopback clients only, or for any client.
#127.0.0.1:submissions inet n  -       y       -       -       smtpd
#submissions     inet  n       -       y       -       -       smtpd
#  -o syslog_name=postfix/submissions
#  -o smtpd_tls_wrappermode=yes
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#     Instead of specifying complex smtpd_<xxx>_restrictions here,
#     specify "smtpd_<xxx>_restrictions=$mua_<xxx>_restrictions"
#     here, and specify mua_<xxx>_restrictions in main.cf (where
#     "<xxx>" is "client", "helo", "sender", "relay", or "recipient").
#  -o smtpd_client_restrictions=
#  -o smtpd_helo_restrictions=
#  -o smtpd_sender_restrictions=
#  -o smtpd_relay_restrictions=
#  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
#628       inet  n       -       y       -       -       qmqpd
pickup    unix  n       -       y       60      1       pickup
cleanup   unix  n       -       y       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
#qmgr     unix  n       -       n       300     1       oqmgr
tlsmgr    unix  -       -       y       1000?   1       tlsmgr
rewrite   unix  -       -       y       -       -       trivial-rewrite
bounce    unix  -       -       y       -       0       bounce
defer     unix  -       -       y       -       0       bounce
trace     unix  -       -       y       -       0       bounce
verify    unix  -       -       y       -       1       verify
flush     unix  n       -       y       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       y       -       -       smtp
relay     unix  -       -       y       -       -       smtp
        -o syslog_name=postfix/$service_name
#       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq     unix  n       -       y       -       -       showq
error     unix  -       -       y       -       -       error
retry     unix  -       -       y       -       -       error
discard   unix  -       -       y       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       y       -       -       lmtp
anvil     unix  -       -       y       -       1       anvil
scache    unix  -       -       y       -       1       scache
postlog   unix-dgram n  -       n       -       1       postlogd
#
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# Many of the following services use the Postfix pipe(8) delivery
# agent.  See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
#
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in main.cf: maildrop_destination_recipient_limit=1
#
maildrop  unix  -       n       n       -       -       pipe
  flags=DRXhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
#
# ====================================================================
#
# Recent Cyrus versions can use the existing "lmtp" master.cf entry.
#
# Specify in cyrus.conf:
#   lmtp    cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
#
# Specify in main.cf one or more of the following:
#  mailbox_transport = lmtp:inet:localhost
#  virtual_transport = lmtp:inet:localhost
#
# ====================================================================
#
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in main.cf: cyrus_destination_recipient_limit=1
#
#cyrus     unix  -       n       n       -       -       pipe
#  flags=DRX user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
#
# ====================================================================
# Old example of delivery via Cyrus.
#
#old-cyrus unix  -       n       n       -       -       pipe
#  flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
#
# ====================================================================
#
# See the Postfix UUCP_README file for configuration details.
#
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
#
# Other external delivery methods.
#
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix -       n       n       -       2       pipe
  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman   unix  -       n       n       -       -       pipe
  flags=FRX user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${user}
submission	inet	n	-	y	-	-	smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may
smtps	inet	n	-	y	-	-	smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may -o smtpd_tls_wrappermode=yes

My /etc/opendkim.conf:

# This is a basic configuration for signing and verifying. It can easily be
# adapted to suit a basic installation. See opendkim.conf(5) and
# /usr/share/doc/opendkim/examples/opendkim.conf.sample for complete
# documentation of available configuration parameters.

Syslog yes
SyslogSuccess		yes
#LogWhy			no

# Common signing and verification parameters. In Debian, the "From" header is
# oversigned, because it is often the identity key used by reputation systems
# and thus somewhat security sensitive.
Canonicalization	relaxed/simple
#Mode			sv
#SubDomains		no
OversignHeaders		From

# Signing domain, selector, and key (required). For example, perform signing
# for domain "example.com" with selector "2020" (2020._domainkey.example.com),
# using the private key stored in /etc/dkimkeys/example.private. More granular
# setup options can be found in /usr/share/doc/opendkim/README.opendkim.
Domain /etc/dkim-domains.txt
Selector 202404
KeyFile /etc/dkim.key

# In Debian, opendkim runs as user "opendkim". A umask of 007 is required when
# using a local socket with MTAs that access the socket as a non-privileged
# user (for example, Postfix). You may need to add user "postfix" to group
# "opendkim" in that case.
UserID			opendkim
UMask			007

# Socket for the MTA connection (required). If the MTA is inside a chroot jail,
# it must be ensured that the socket is accessible. In Debian, Postfix runs in
# a chroot in /var/spool/postfix, therefore a Unix socket would have to be
# configured as shown on the last line below.
Socket inet:8891@127.0.0.1
#Socket			inet:8891@localhost
#Socket			inet:8891
#Socket			local:/var/spool/postfix/opendkim/opendkim.sock

PidFile			/run/opendkim/opendkim.pid

# Hosts for which to sign rather than verify, default is 127.0.0.1. See the
# OPERATION section of opendkim(8) for more information.
#InternalHosts		192.168.0.0/16, 10.0.0.0/8, 172.16.0.0/12

# The trust anchor enables DNSSEC. In Debian, the trust anchor file is provided
# by the package dns-root-data.
TrustAnchorFile		/usr/share/dns/root.key
#Nameservers		127.0.0.1

My /etc/resolv.conf:

nameserver 127.0.0.1
nameserver 8.8.8.8
nameserver 8.8.4.4

I see from https://forum.directadmin.com/threads/configure-dkim-for-host-domain-com-email-addresses.59809/ that "The default DNS zone host.domain.com that is added on install doesn't have the options out of the box to enable DKIM," and DirectAdmin has created a separate script to enable it. Is there something similar for Virtualmin? If not, any idea why Postfix won't properly attach this required information to command line emails? Thanks.

Operating system 	Debian Linux 12
Webmin version 	2.105
Usermin version 	2.005
Virtualmin version 	7.10.0
BIND version 	9.18
Postfix version 	3.7.10
@chris001
Copy link

DirectAdmin has created a separate script to enable it.

Would you copy paste the separate script DirectAdmin created.

@c-prompt
Copy link
Author

DirectAdmin has created a separate script to enable it.

Would you copy paste the separate script DirectAdmin created.

Apologies but, as far as I can tell, there's an "old script" and a "new script," both of which I don't have access to as I don't use DirectAdmin ;) But the detailed instructions are at https://docs.directadmin.com/other-hosting-services/exim/configuring-exim.html#dkim-installation-guide and utilize Exim. The new method script is referenced at that URL so perhaps someone who has access to DirectAdmin can provide more details. The old method seems to be referenced in https://forum.directadmin.com/threads/configure-dkim-for-host-domain-com-email-addresses.59809/post-306301 and just refers to:

Just run:
/usr/local/directadmin/scripts/dkim_create.sh host.domain.com

But https://forum.directadmin.com/threads/configure-dkim-for-host-domain-com-email-addresses.59809/post-309460 also notes:

DKIM record for a hostname as well as a separate DNS zone for a hostname might be required. Directadmin will add a public DKIM key for a hostname only into /var/named/hostname.db file, i.e. DNS zone created for the hostname.

DirectAdmin still sends emails from root@hostname, admin@hostname, etc. So a DKIM record for hostname is good option to get a higher trust level when sending emails.

And as of now a DKIM for hostname can be created only using the old way, see the step #4

Code:
cd /usr/local/directadmin/scripts
./dkim_create.sh $(hostname -f)

Run the two commands without modifications, as they are shown. There is nothing to replace.

@chris001
Copy link

chris001 commented Apr 17, 2024

  1. Add postfix user to opendkim group.
sudo gpasswd -a postfix opendkim
  1. Edit OpenDKIM main configuration file.
sudo nano /etc/opendkim.conf
  1. Find the following line.
Syslog               yes
  1. By default, OpenDKIM logs will be saved in /var/log/mail.log file. Add the following line so OpenDKIM will generate more detailed logs for debugging.
Logwhy               yes
  1. Send a mail.
echo "Test message - you know the drill" | mail -r "root <root@host.sub.myservername.com>" -s "Test Message" test-d1b79814@appmaildev.com
  1. Look at the log for the reason why OpenDKIM didn't sign the mail.
tail -100 /var/log/mail.log
  1. Test your DKIM key

Example: Ubuntu server

sudo opendkim-testkey -d host.sub.myservername.com -s default -vvv

If everything is OK, you will see key OK in the command output.

opendkim-testkey: using default configfile /etc/opendkim.conf
opendkim-testkey: checking key 'yourselector._domainkey.host.sub.myservername.com'
opendkim-testkey: key secure
opendkim-testkey: key OK

Note that your DKIM record may need some time to propagate to the Internet's caching DNS servers. Depending on the domain registrar you use, your DNS record might be propagated instantly, or it might take up to 24 hours to propagate. You can go to https://www.dmarcanalyzer.com/dkim/dkim-check/ , enter your domain's DKIM selector, and enter your domain name, to check your DKIM record's DNS propagation time.

If you see Key not secure in the command output, don’t panic, this is because DNSSEC isn’t enabled on your domain name. There’s no need to worry about Key not secure.

If you see the query timed out error, you need to comment out the following line in /etc/opendkim.conf file and restart opendkim.service.

TrustAnchorFile       /usr/share/dns/root.key

sudo systemctl restart opendkim.service

  1. Postfix has TWO sets of mail filters: filters that are used for SMTP mail only (specified with the smtpd_milters parameter), and filters for non-SMTP mail (specified with the non_smtpd_milters parameter). The non-SMTP filters are primarily for local submissions. Verify the non_smtpd_milters has your DKIM milter listed.

  2. Post back your results.

@c-prompt
Copy link
Author

Wow! Thanks for the quick response!

Results in mail.log:

2024-04-16T21:10:54.254752-04:00 host postfix/pickup[833452]: 3DF1A66E9A: uid=0 from=<root@host.sub.myservername.com>
2024-04-16T21:10:54.275747-04:00 host postfix/cleanup[837253]: 3DF1A66E9A: message-id=<20240417011054.3DF1A66E9A@host.sub.myservername.com>
2024-04-16T21:10:54.361224-04:00 host postfix/qmgr[814060]: 3DF1A66E9A: from=<root@host.sub.myservername.com>, size=542, nrcpt=1 (queue active)
2024-04-16T21:10:56.847767-04:00 host postfix/smtp[837255]: 3DF1A66E9A: to=<test-e2ebd39f@appmaildev.com>, relay=appmaildev.com[13.76.39.245]:25, delay=2.7, delays=0.22/0.03/1.7/0.73, dsn=2.6.0, status=sent (250 2.6.0  <20240417011054.3DF1A66E9A@host.sub.myservername.com> Queued mail for delivery)
2024-04-16T21:10:56.848463-04:00 host postfix/qmgr[814060]: 3DF1A66E9A: removed

Detailed report from appmaildev.com (all same issues):

============================================================================
This is SPF/DKIM/DMARC/RBL report generated by a test tool provided 
	by AdminSystem Software Limited.

Any problem, please contact support@emailarchitect.net
============================================================================
Report-Id: e2ebd39f
Sender: <root@host.sub.myservername.com>
Header-From: <root@host.sub.myservername.com>
HELO-Domain: host.sub.myservername.com
Source-IP: server_ip_address
SSL/TLS: TLS secured
Validator-Version: 1.19
============================================================================
Original email header:

x-sender: root@host.sub.myservername.com
x-receiver: test-e2ebd39f@appmaildev.com
Received: from host.sub.myservername.com ([server_ip_address]) by appmaildev.com over TLS secured channel with Microsoft SMTPSVC(8.5.9600.16384);
	 Wed, 17 Apr 2024 01:10:55 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=host.sub.myservername.com; s=202404; t=1713316254;
	bh=e7R0pFzcLDW2cd86kMjeWyx8JMvgFlzFN7EZwVhpr+Y=;
	h=From:To:Subject:Date:From;
	b=fkUJr6ts8CvRgn6UvlpD07BWYqiKCeCP5UbvjiOvVoruFZdVwCN7XbmCKZtXZDlUB
	 kcYU6Lf5/qghXtpDNL/FS9kG40eQKzs56jEtjUPHvX+fJ2gUkoCodluovEjIUTcOwY
	 blCX0xArjbhgaKLdaO8OD3ynq6MwxwWIikw97hUOP12D1Z6yNKc0mIIPMmAiuAt1cL
	 A3KRQ9loXfjUl20sBJ/IAKt4n00KYKLtDLTJQI84Y9VPABjLxMB/89eQEnPDBLjJEx
	 wOEUCs5jfsSMBhAUsLoa6jSWRks3r5ROHOFtar4xBxK1F393j5T9LjhVAgFzBwDfjy
	 B8t3SdIIrTxLg==
Received: by host.sub.myservername.com (Postfix, from userid 0)
	id 3DF1A66E9A; Tue, 16 Apr 2024 21:10:54 -0400 (EDT)
From: root <root@host.sub.myservername.com>
To: test-e2ebd39f@appmaildev.com
Subject: Test Message
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Message-Id: <20240417011054.3DF1A66E9A@host.sub.myservername.com>
Date: Tue, 16 Apr 2024 21:10:54 -0400 (EDT)
Return-Path: root@host.sub.myservername.com
X-OriginalArrivalTime: 17 Apr 2024 01:10:56.0428 (UTC) FILETIME=[19C45EC0:01DA9064]

============================================================================
SPF: None
============================================================================

Sender-IP: server_ip_address
Sender-Domain-Helo-Domain: host.sub.myservername.com

Query TEXT record from DNS server for: host.sub.myservername.com
Exception: No records found for given DNS query

============================================================================
DKIM: permerror
============================================================================

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=host.sub.myservername.com; s=202404; t=1713316254;
	bh=e7R0pFzcLDW2cd86kMjeWyx8JMvgFlzFN7EZwVhpr+Y=;
	h=From:To:Subject:Date:From;
	b=fkUJr6ts8CvRgn6UvlpD07BWYqiKCeCP5UbvjiOvVoruFZdVwCN7XbmCKZtXZDlUB
	 kcYU6Lf5/qghXtpDNL/FS9kG40eQKzs56jEtjUPHvX+fJ2gUkoCodluovEjIUTcOwY
	 blCX0xArjbhgaKLdaO8OD3ynq6MwxwWIikw97hUOP12D1Z6yNKc0mIIPMmAiuAt1cL
	 A3KRQ9loXfjUl20sBJ/IAKt4n00KYKLtDLTJQI84Y9VPABjLxMB/89eQEnPDBLjJEx
	 wOEUCs5jfsSMBhAUsLoa6jSWRks3r5ROHOFtar4xBxK1F393j5T9LjhVAgFzBwDfjy
	 B8t3SdIIrTxLg==
Signed-by: root@host.sub.myservername.com
Expected-Body-Hash: e7R0pFzcLDW2cd86kMjeWyx8JMvgFlzFN7EZwVhpr+Y=
Current Utc timestamp: 2024-04-17T01:10:56.726; Signature timestamp: 2024-04-17T01:10:54.000

Canonicalized header: from:root <root@host.sub.myservername.com>
to:test-e2ebd39f@appmaildev.com
subject:Test Message
date:Tue, 16 Apr 2024 21:10:54 -0400 (EDT)


DKIM-Result: permerror (no key)

============================================================================
DMARC: permerror
============================================================================

_dmarc.myservername.com: v=DMARC1; p=none; pct=100
Received-SPF: none (appmaildev.com: domain of root@host.sub.myservername.com does not designates permitted sender) client-ip=server_ip_address
Authentication-Results: appmaildev.com;
    dkim=permerror (no key) header.d=host.sub.myservername.com;
    spf=none (appmaildev.com: domain of root@host.sub.myservername.com does not designates permitted sender) client-ip=server_ip_address;
    dmarc=permerror (adkim=r aspf=r p=none) header.from=host.sub.myservername.com;

============================================================================
DomainKey: none
============================================================================

DomainKey-Result: none (no signature)
If DKIM result is passed, you can ignore DomainKey result: none
Notice: DomainKey is obsoleted standard, the new standard is DKIM.

============================================================================
PTR: ExistsRecord
============================================================================

Sender-IP: server_ip_address
Query 189.66.45.38.in-addr.arpa
Host: host.sub.myservername.com

============================================================================
RBL: NotListed
============================================================================

bl.spamcop.net:Not Listed (OK) - http://bl.spamcop.net 
cbl.abuseat.org:Not Listed (OK) - http://cbl.abuseat.org 
b.barracudacentral.org:Not Listed (OK) - http://www.barracudacentral.org/rbl/removal-request 
dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
http.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
dul.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
misc.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
smtp.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
socks.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
spam.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
web.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
zombie.dnsbl.sorbs.net:Not Listed (OK) - http://www.sorbs.net 
pbl.spamhaus.org:Not Listed (OK) - http://www.spamhaus.org/pbl/ 
sbl.spamhaus.org:Not Listed (OK) - http://www.spamhaus.org/sbl/ 
xbl.spamhaus.org:Not Listed (OK) - http://www.spamhaus.org/xbl/ 
zen.spamhaus.org:Not Listed (OK) - http://www.spamhaus.org/zen/ 
ubl.unsubscore.com:Not Listed (OK) - http://www.lashback.com/blacklist/ 
rbl.spamlab.com:Not Listed (OK) - http://tools.appriver.com/index.aspx?tool=rbl 
dyna.spamrats.com:Not Listed (OK) - http://www.spamrats.com 
noptr.spamrats.com:Not Listed (OK) - http://www.spamrats.com 
spam.spamrats.com:Not Listed (OK) - http://www.spamrats.com 
dnsbl.inps.de:Not Listed (OK) - http://dnsbl.inps.de/index.cgi?lang=en 
drone.abuse.ch:Not Listed (OK) - http://dnsbl.abuse.ch 
httpbl.abuse.ch:Not Listed (OK) - http://dnsbl.abuse.ch 
korea.services.net:Not Listed (OK) - http://korea.services.net 
short.rbl.jp:Not Listed (OK) - http://www.rbl.jp 
virus.rbl.jp:Not Listed (OK) - http://www.rbl.jp 
spamrbl.imp.ch:Not Listed (OK) - http://antispam.imp.ch 
wormrbl.imp.ch:Not Listed (OK) - http://antispam.imp.ch 
virbl.bit.nl:Not Listed (OK) - http://virbl.bit.nl  
rbl.suresupport.com:Not Listed (OK) - http://suresupport.com/postmaster 
dsn.rfc-ignorant.org:Not Listed (OK) - http://www.rfc-ignorant.org/policy-dsn.php 
spamguard.leadmon.net:Not Listed (OK) - http://www.leadmon.net/SpamGuard/ 
dnsbl.tornevall.org:Not Listed (OK) - http://opm.tornevall.org 
netblock.pedantic.org:Not Listed (OK) - http://pedantic.org 
multi.surbl.org:Not Listed (OK) - http://www.surbl.org 
ix.dnsbl.manitu.net:Not Listed (OK) - http://www.dnsbl.manitu.net 
tor.dan.me.uk:Not Listed (OK) - http://www.dan.me.uk/dnsbl 
rbl.efnetrbl.org:Not Listed (OK) - http://rbl.efnetrbl.org 
dnsbl.dronebl.org:Not Listed (OK) - http://www.dronebl.org 
access.redhawk.org:Not Listed (OK) - http://www.redhawk.org/index.php?option=com_wrapper&Itemid=33 
db.wpbl.info:Not Listed (OK) - http://www.wpbl.info 
rbl.interserver.net:Not Listed (OK) - http://rbl.interserver.net 
query.senderbase.org:Not Listed (OK) - http://www.senderbase.org/about 
bogons.cymru.com:Not Listed (OK) - http://www.team-cymru.org/Services/Bogons/ 
csi.cloudmark.com:Not Listed (OK) - http://www.cloudmark.com/en/products/cloudmark-sender-intelligence/index 

cbl.anti-spam.org.cn:DnsTimeout - http://www.anti-spam.org.cn/?Locale=en_US 
cdl.anti-spam.org.cn:DnsTimeout - http://www.anti-spam.org.cn/?Locale=en_US 


============================================================================
Original message source
============================================================================
x-sender: root@host.sub.myservername.com
x-receiver: test-e2ebd39f@appmaildev.com
Received: from host.sub.myservername.com ([server_ip_address]) by appmaildev.com over TLS secured channel with Microsoft SMTPSVC(8.5.9600.16384);
	 Wed, 17 Apr 2024 01:10:55 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=host.sub.myservername.com; s=202404; t=1713316254;
	bh=e7R0pFzcLDW2cd86kMjeWyx8JMvgFlzFN7EZwVhpr+Y=;
	h=From:To:Subject:Date:From;
	b=fkUJr6ts8CvRgn6UvlpD07BWYqiKCeCP5UbvjiOvVoruFZdVwCN7XbmCKZtXZDlUB
	 kcYU6Lf5/qghXtpDNL/FS9kG40eQKzs56jEtjUPHvX+fJ2gUkoCodluovEjIUTcOwY
	 blCX0xArjbhgaKLdaO8OD3ynq6MwxwWIikw97hUOP12D1Z6yNKc0mIIPMmAiuAt1cL
	 A3KRQ9loXfjUl20sBJ/IAKt4n00KYKLtDLTJQI84Y9VPABjLxMB/89eQEnPDBLjJEx
	 wOEUCs5jfsSMBhAUsLoa6jSWRks3r5ROHOFtar4xBxK1F393j5T9LjhVAgFzBwDfjy
	 B8t3SdIIrTxLg==
Received: by host.sub.myservername.com (Postfix, from userid 0)
	id 3DF1A66E9A; Tue, 16 Apr 2024 21:10:54 -0400 (EDT)
From: root <root@host.sub.myservername.com>
To: test-e2ebd39f@appmaildev.com
Subject: Test Message
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Message-Id: <20240417011054.3DF1A66E9A@host.sub.myservername.com>
Date: Tue, 16 Apr 2024 21:10:54 -0400 (EDT)
Return-Path: root@host.sub.myservername.com
X-OriginalArrivalTime: 17 Apr 2024 01:10:56.0428 (UTC) FILETIME=[19C45EC0:01DA9064]

Test message - you know the drill
============================================================================

Result of DKIM test:

root@server_ip_address ~ $ sudo opendkim-testkey -d host.sub.myservername.com -s default -vvv
opendkim-testkey: using default configfile /etc/opendkim.conf
opendkim-testkey: key loaded from /etc/dkim.key
opendkim-testkey: checking key 'default._domainkey.host.sub.myservername.com'
opendkim-testkey: 'default._domainkey.host.sub.myservername.com' query timed out

So I commented out TrustAnchorFile and sudo systemctl restart opendkim.service, which still timed out:

root@server_ip_address ~ $ sudo systemctl restart opendkim.service
root@server_ip_address ~ $ sudo opendkim-testkey -d host.sub.myservername.com -s default -vvv
opendkim-testkey: using default configfile /etc/opendkim.conf
opendkim-testkey: key loaded from /etc/dkim.key
opendkim-testkey: checking key 'default._domainkey.host.sub.myservername.com'
opendkim-testkey: 'default._domainkey.host.sub.myservername.com' query timed out

The results of https://www.mimecast.com/products/dmarc-analyzer/dkim-check/ are: "Error: Unable to find a DKIM record." Thus, I will keep trying over the next 24 hours to see if there are any changes.

As indicated above, the milters are the same; please let me know if you think non_smtpd_milters should be otherwise:

smtpd_milters = inet:127.0.0.1:8891
non_smtpd_milters = inet:127.0.0.1:8891

@chris001
Copy link

chris001 commented Apr 17, 2024

  1. SPF policies are not automatically inherited by subdomains! When you send emails from subdomains, you should configure SPF DNS records for each of these subdomains by adding an SPF DNS entry for each subdomain. To solve this easily, you make an SPF record for the subdomain, and set it so the subdomain is allowed to send from the same origins as the main domain, with: v=spf1 ?include:myservername.com ~all.

  2. DKIM Alignment hinges on the domain in your "FROM" header matching the domain used in the DKIM signature (d=myservername.com). This uses a relaxed format by default which means that a sub-domain would partially match and therefore align OK. If this alignment setting is changed to strict in your DMARC record, then the domain (or subdomain) in the From header must match the d=xxxxxxxxx.tld domain in the DKIM signature exactly.

  3. When you don't create DMARC policies for subdomains, they inherit the parent domain's DMARC policy!

@c-prompt
Copy link
Author

c-prompt commented Apr 17, 2024

https://www.mimecast.com/products/dmarc-analyzer/dkim-check/ still shows "Error: Unable to find a DKIM record." (Edit: I am checking for DKIM Selector 202404 of host.sub.myservername.com; however, 202404 does show up for sub.myservername.com).

Not sure I understand: are you suggesting there's something wrong in the DNS records? Because, as I mentioned, all other emails are being sent properly from, e.g., user_name1@sub.myservername.com, user_name2@sub.myservername.com, etc (EDIT: which are sent via host.sub.myservername.com). It is only when an email is generated by the mail command line that it fails.

$ttl 38400
sub.myservername.com.	IN	SOA	host.sub.myservername.com. root.host.sub.myservername.com. (
			2024041700
			3600
			600
			1209600
			3600 )
sub.myservername.com.	IN	A	server_ip_address
localhost.sub.myservername.com.	IN	A	127.0.0.1
sub.myservername.com.	IN	MX	5 host.sub.myservername.com.
sub.myservername.com.	IN	TXT	"v=spf1 a mx a:host.sub.myservername.com mx:host.sub.myservername.com ip4:server_ip_address include:host.sub.myservername.com -all"
host.sub.myservername.com.	IN	A	server_ip_address
sub.myservername.com.	IN	CAA	0 issuewild letsencrypt.org
_dmarc.sub.myservername.com.	IN	TXT	"v=DMARC1; p=none; pct=100"
202404._domainkey.sub.myservername.com.	IN	TXT	( "v=DKIM1; k=rsa; t=s; p=lots_of_characters" "lots_of_characters" "lots_of_characters" "lots_of_characters" "lots_of_characters" "lots_of_characters" )

EDIT: Here's the header from an email sent without issue:

Return-Path: <root@host.sub.myservername.com>
X-Original-To: my_email@email_address.com
Delivered-To: my_email.email_address@host.myservername.com
Received: from host.sub.myservername.com (host.sub.myservername.com [server_ip_address])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
	(No client certificate requested)
	by host.myservername.com (Postfix) with ESMTPS id 74CA76716C
	for <my_email@email_address.com>; Wed, 17 Apr 2024 06:25:23 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=sub.myservername.com; s=202404; t=1713349520;
	bh=TMxbSuJJovdM312hC1AscBqJLa/PeFEvTh4mUc/d7HQ=;
	h=To:From:Subject:Date:From;
	b=Rrhg5e3JbMcL6a1VytggkFh+0Dnj7Ta3ijA57E6sb7jx5S3EVWbJ9ml3fz2QNcaDK
	 gBjFFdvfTrBfK6WAebyQnRcTXS0rNfNFs5/BhqRI1+8AX++54XoBhoHnTZNkJTmPsF
	 MrRU9J6Z989ES3RUoCbfOzQRe4mpCw6u1OVQbKLyU/K6ZxVlSSR5MYZpIvIUYg5hZG
	 u3qEEYmjuVJg5iAh5mv6jYWkVmsfB3WiLdoj/uuX4a/maeSQu/QDCnsQkHqA+ShXrJ
	 nFkCTBLUhURPF/8KjXSi/bqL0U9vpAv0KXF1MdYHThqEEQF9+W5GKB/zcaaDXodET+
	 OdrIYL67ZLtwA==
Received: by host.sub.myservername.com (Postfix, from userid 0)
	id 54496673CB; Wed, 17 Apr 2024 06:25:20 -0400 (EDT)
To: my_email@email_address.com
From: root@sub.myservername.com
Subject: logwatch
Auto-Submitted: auto-generated
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset="UTF-8"
Message-Id: <20240417102520.54496673CB@host.sub.myservername.com>
Date: Wed, 17 Apr 2024 06:25:14 -0400 (EDT)

@c-prompt
Copy link
Author

c-prompt commented Apr 17, 2024

Just to further clarify: this is the header of a cron email from a script that failed which was rejected by Gmail:

Return-Path: <root@host.sub.myservername.com>
Received: by host.sub.myservername.com (Postfix)
	id C98AF673CA; Tue, 16 Apr 2024 14:39:03 -0400 (EDT)
Delivered-To: root@host.sub.myservername.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=host.sub.myservername.com; s=202404; t=1713292743;
	bh=cipxyYXcbtkYWioFHjLc9hlK5JFZnaqe3K3uaeT6wz4=;
	h=From:To:Subject:Date:From;
	b=SR4wOndoKLNOKcVsLKVoE9YhaZZfE0sQ3C+jbj2ir2gtPchQO6OHnYebIzOmLIpDj
	 QZvTP3iReEpylYzn5MokuVpvcDw+jCCcwFy8mx5uI8WoDaqkIi/8KXB2L+X1A7H1Ri
	 My+t1E97YQQlREmcLtCSeKT5+AkHJBK5i2xkrHSlzCOqkzkyy2u1r57m9v6CxhqN42
	 pF87J58dWu+cqncK0IibA/7TllCy/LvySsS1wP+KBxdqb7ptGXW+5inxCyX8GrL74+
	 yDMvye1xd8GzaFCEEgR9WW13e29RtowC12YhMOfrueBEk4JGBxtzUSWb9WTmao8zin
	 XOx8gR4qg0eoQ==
Received: by host.sub.myservername.com (Postfix, from userid 0)
	id ADE6E66E55; Tue, 16 Apr 2024 14:39:03 -0400 (EDT)
From: root@host.sub.myservername.com (Cron Daemon)
To: root@host.sub.myservername.com
Subject: Cron <root@host> .sh failed
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
Message-Id: <20240416183903.ADE6E66E55@host.sub.myservername.com>
Date: Tue, 16 Apr 2024 14:39:03 -0400 (EDT)

This is why Gmail said it rejected the email:

Reporting-MTA: dns; host.sub.myservername.com
X-Postfix-Queue-ID: C98AF673CA
X-Postfix-Sender: rfc822; root@host.sub.myservername.com
Arrival-Date: Tue, 16 Apr 2024 14:39:03 -0400 (EDT)

Final-Recipient: rfc822; my_email@gmail.com
Original-Recipient: rfc822;root@host.sub.myservername.com
Action: failed
Status: 5.7.26
Remote-MTA: dns; gmail-smtp-in.l.google.com
Diagnostic-Code: smtp; 550-5.7.26 This mail has been blocked because the sender
    is unauthenticated. 550-5.7.26 Gmail requires all senders to authenticate
    with either SPF or DKIM. 550-5.7.26  550-5.7.26  Authentication results:
    550-5.7.26  DKIM = did not pass 550-5.7.26  SPF
    [host.sub.myservername.com] with ip: [server_ip_address 550-5.7.26
    ] = did not pass 550-5.7.26  550-5.7.26  For instructions on setting up
    authentication, go to 550 5.7.26
    https://support.google.com/mail/answer/81126#authentication
    if7-20020a0561022c8700b0047b875f02d0si1005655vsb.248 - gsmtp

@chris001
Copy link

The error messages says it requires either DKIM or SPF, or both, to pass, for host.sub.myservername.com.

This mail has been blocked because the sender
    is unauthenticated. 550-5.7.26 Gmail requires all senders to authenticate
    with either SPF or DKIM. 550-5.7.26  550-5.7.26  Authentication results:
    550-5.7.26  DKIM = did not pass 550-5.7.26  SPF
    [host.sub.myservername.com] with ip: [server_ip_address 550-5.7.26
    ] = did not pass 550-5.7.26  550-5.7.26  For instructions on setting up
    authentication, go to 550 5.7.26
    https://support.google.com/mail/answer/81126#authentication
    if7-20020a0561022c8700b0047b875f02d0si1005655vsb.248 - gsmtp

To enable SPF for host.sub.myservername.com is easy, could you add host IN TXT "v=spf1 ?include:sub.myservername.com ~all" to the sub.myservername.com zone, Save, Reload BIND9, and try again?

@c-prompt
Copy link
Author

Ah, that's my bad. I had added (which didn't work):

host.sub.myservername.com. IN TXT "v=spf1 a mx a:host.sub.myservername.com mx:host.sub.myservername.com ip4:server_ip_address include:host.sub.myservername.com -all"

Just to confirm then: it should be host (without the period at the end) instead of host. or host.sub.myservername.com., correct (because all my other DNS records have a period at the end)?

@chris001
Copy link

Yes host will be a subdomain relative to the current zone/domain, which is sub.myservername.com. Or you can fully write it out as host.sub.myservername.com. with the ending period meaning it's the full domain name, not a relative subdomain of the current zone.

One error in your SPF record is include:host.sub.myservername.com you're telling it to include itself which is an infinite loop and gets you nothing. You should include the parent domain or any other domain, include:sub.myservername.com

@c-prompt
Copy link
Author

One error in your SPF record is include:host.sub.myservername.com you're telling it to include itself which is an infinite loop and gets you nothing. You should include the parent domain or any other domain, include:sub.myservername.com

Ah, that makes sense and resulted from a straight copy/paste without close evaluation. But it makes me curious and shows my lack of understanding as to why the original TXT doesn't work. I thought include:host.sub.myservername.com -all in my original TXT record:

sub.myservername.com. IN TXT "v=spf1 a mx a:host.sub.myservername.com mx:host.sub.myservername.com ip4:server_ip_address include:host.sub.myservername.com -all"

...was the proper way to include host. You're saying you need a separate TXT record for host.

And apologies for being dense but I still don't understand then how to add a DKIM for host.sub.myservername.com.

Let me know if you think it makes sense to open a new feature request that automatically adds DKIM, SPF, and DMARC records like DirectAdmin as I've got to imagine I'm not the only one who sees Gmail rejecting emails.

I'll report back once propagation occurs.

@c-prompt
Copy link
Author

Interesting... so SPF now comes back as "Neutral" when verifying through https://www.appmaildev.com/:

SPF-Record: v=spf1 ?include:sub.myservername.com ~all
Sender-IP: server_ip_address
Sender-Domain-Helo-Domain: host.sub.myservername.com

Query TEXT record from DNS server for: host.sub.myservername.com
[TXT]: v=spf1 ?include:sub.myservername.com ~all
Parsing SPF record: v=spf1 ?include:sub.myservername.com ~all

Mechanisms: v=spf1

Mechanisms: ?include:sub.myservername.com
Testing mechanism include:sub.myservername.com
Query TEXT record from DNS server for: sub.myservername.com
[TXT]: v=spf1 a mx a:host.sub.myservername.com mx:host.sub.myservername.com ip4:server_ip_address include:host.sub.myservername.com -all
Parsing SPF record: v=spf1 a mx a:host.sub.myservername.com mx:host.sub.myservername.com ip4:server_ip_address include:host.sub.myservername.com -all

Mechanisms: v=spf1

Mechanisms: a
Testing mechanism a
Query A record from DNS server for: sub.myservername.com
[A]: server_ip_address
Testing CIDR: source=server_ip_address;  server_ip_address/128
a hit, Qualifier: +
include:sub.myservername.com hit, Qualifier: ?

But Gmail is still unhappy:

Reporting-MTA: dns; host.sub.myservername.com
X-Postfix-Queue-ID: 4B3AF673CD
X-Postfix-Sender: rfc822; root@host.sub.myservername.com
Arrival-Date: Wed, 17 Apr 2024 18:01:02 -0400 (EDT)

Final-Recipient: rfc822; my_email@gmail.com
Original-Recipient: rfc822;root@host.sub.myservername.com
Action: failed
Status: 5.7.26
Remote-MTA: dns; gmail-smtp-in.l.google.com
Diagnostic-Code: smtp; 550-5.7.26 This mail has been blocked because the sender
    is unauthenticated. 550-5.7.26 Gmail requires all senders to authenticate
    with either SPF or DKIM. 550-5.7.26  550-5.7.26  Authentication results:
    550-5.7.26  DKIM = did not pass 550-5.7.26  SPF
    [host.sub.myservername.com] with ip: [server_ip_address 550-5.7.26
    ] = did not pass 550-5.7.26  550-5.7.26  For instructions on setting up
    authentication, go to 550 5.7.26
    https://support.google.com/mail/answer/81126#authentication
    ik7-20020a0561025f0700b00479d6d18180si30950vsb.534 - gsmtp

@chris001
Copy link

If you remove the leading ? in front of include, it will change from neutral to allow, and should pass Gmail with SPF passing. (The neutral is a high security setting.)

@chris001
Copy link

Since each subdomain has to have its very own SPF record, there's no way to state in the sub.myservername.com that its subdomain host was also allowed to send mail from host. Rather, host needs its own SPF record stating which servers are allowed to send mail from host.

@c-prompt
Copy link
Author

And Gmail is now (kind of) happy as it passes SPF and DMARC - thank you!

Delivered-To: my_email@gmail.com
Received: by 2002:a05:6520:3843:b0:28e:b8fb:27f9 with SMTP id c3csp1926078lkx;
        Wed, 17 Apr 2024 15:51:03 -0700 (PDT)
X-Forwarded-Encrypted: i=2; AJvYcCXUpqa8Iz8amSkh3tsx+d61SbFlov0+6i0z20vn8H7iBByHUI4p74LTKsJygXatqMX+h6Yd7q3BIIi+Jd3rpMJKJyAANFsD7gzwwQCz
X-Google-Smtp-Source: AGHT+IG3M/9a4nPZQzIzSQ3AzUBiznU1w9ParmFW2UoZ+h2LQYE9LP4YSdaxQyJwReD9rCjO8y+P
X-Received: by 2002:a05:6122:310c:b0:4d8:7d49:18fe with SMTP id cg12-20020a056122310c00b004d87d4918femr899582vkb.4.1713394263189;
        Wed, 17 Apr 2024 15:51:03 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; t=1713394263; cv=none;
        d=google.com; s=arc-20160816;
        b=P6CpXbzi3asV4NGhEtuQxfdP40wuAOVRqev08Q86zFxJWe0KS794D9p6HPQRtXKQxU
         nqgmVdnw+MynVju9Xls4zANq61o1f/eQshpLOGbjMfOWc3iczJdYJVFtzM5BdySXObCY
         vr9Z881eQT1+cHAIp0EoBcgVfh7FZOj3E4p14BnLu/KOdQvdStrUToipBdPrpnXbFHcq
         pXffNqwVYxzABYpYs5umNIhfqE3FrlHiFx1+leiUi7RpWQrgYEkUF/xzmiMWhpstNods
         jvw+AYkPXk7a+Sp3oAn5qVRHCzr59C5ukL/TXPs26dLPmYR9YPpnfF7gzvoMqNUADJ30
         N43A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=date:message-id:content-transfer-encoding:mime-version:subject:to
         :from:dkim-signature:delivered-to;
        bh=DP7HZiz68kwexRJ3V8NvAQThyywT/TtJI0BXBSNlxdw=;
        fh=Le2JzKC5blRy88uoK5BTpJgVlJYfuGBm/tvURDS8Obw=;
        b=0RmycbDpZp1dl3aJ9siSUGytjgZGCwPqe9Kge4HWIPwyWtWbXS7oZy/NsTuWECAe/C
         k8guHEO5FSErl1wXesx6vGJCSAqGiDFcsd8irkNYzaqaGzcOqbHF56REo7REqcI8dEk2
         zDriiv5IUKW+6K+3YeLFumt5o0TWS2hEcZXyZE7KwR1bLqghW8oNqP9JURUVdxYlBPQi
         dKYTHGEP0N3e8RFI3CctKJpQ29lQ754k76lw8GxFHFo4pAVmTYkLW3cnwlfXOFxSg0vM
         X3p8KZurnuIsGbfgSUuiHcX7xaoKzlFdAPVddujV4gJcmxOx1d5HU5iDwlsngtLWeWI+
         Y3bA==;
        dara=google.com
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=temperror (no key for signature) header.i=@host.sub.myservername.com header.s=202404 header.b=mp5ZpHuN;
       spf=pass (google.com: domain of root@host.sub.myservername.com designates server_ip_address as permitted sender) smtp.mailfrom=root@host.sub.myservername.com;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=myservername.com
Return-Path: <root@host.sub.myservername.com>
Received: from host.sub.myservername.com (host.sub.myservername.com. [server_ip_address])
        by mx.google.com with ESMTPS id fe2-20020a056130188200b007e788c26cdasi43834uab.107.2024.04.17.15.51.02
        for <my_email@gmail.com>
        (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
        Wed, 17 Apr 2024 15:51:02 -0700 (PDT)
Received-SPF: pass (google.com: domain of root@host.sub.myservername.com designates server_ip_address as permitted sender) client-ip=server_ip_address;
Authentication-Results: mx.google.com;
       dkim=temperror (no key for signature) header.i=@host.sub.myservername.com header.s=202404 header.b=mp5ZpHuN;
       spf=pass (google.com: domain of root@host.sub.myservername.com designates server_ip_address as permitted sender) smtp.mailfrom=root@host.sub.myservername.com;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=myservername.com
Received: by host.sub.myservername.com (Postfix)
	id C7F2A66E94; Wed, 17 Apr 2024 18:51:02 -0400 (EDT)
Delivered-To: root@host.sub.myservername.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=host.sub.myservername.com; s=202404; t=1713394262;
	bh=DP7HZiz68kwexRJ3V8NvAQThyywT/TtJI0BXBSNlxdw=;
	h=From:To:Subject:Date:From;
	b=mp5ZpHuNWpA8S/pLe8PZ743a4wqKhCYb4HQ1TJ9yv23TAScjg2DKFlVPcrrMKW3F6
	 zhHA92y75/CupYdNSAZRrkUwDPlXiGx1qSKDGzVXXHC+RdRmwqPUWcDQKr/r3YlPoe
	 GDqe4gLxl8Xv4YZ8VnRVPYDOJLMI5L7NrqvfUISSXpSzT+rhLX6KUT7qvtIQqKuMZ/
	 ZiQ82Fyj7BcZOTZOsIXHonXlSz7mbqXveouPUYHnBgIjb3ijyLJhCBu8iv34kUoIvj
	 cTPnnAIDfTGV3dt2RzWfAI7jn3sPBua2OGL9FUIShMZnBqzFlGFAxrdffJNjyNWYjA
	 p4Ce3427fykZw==
Received: by host.sub.myservername.com (Postfix, from userid 0)
	id B277C66E89; Wed, 17 Apr 2024 18:51:02 -0400 (EDT)
From: root@host.sub.myservername.com (Cron Daemon)
To: root@host.sub.myservername.com
Subject: Cron <root@host> Script fail
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
Message-Id: <20240417225102.B277C66E89@host.sub.myservername.com>
Date: Wed, 17 Apr 2024 18:51:02 -0400 (EDT)

@chris001
Copy link

chris001 commented Apr 18, 2024

Thank you for plowing thru til the solution! Bonus, To make your email pass DKIM also, just go to virtualmin and add "host.sub.myservername.com" as a virtual server or sub server, enable DKIM on it, and while you're at it, enable SPF and DMARC, so that all three of these features' DNS entries will be managed. For the DKIM key, although they don't expire, they really should be rotated periodically - every 12 months is good - this would be a good nice-to-have feature request / new issue to open, to add a setting to have virtualmin auto rotate the DKIM key(s) every X months.

@c-prompt
Copy link
Author

c-prompt commented Apr 18, 2024

One of the troubleshooting steps I tried before the OP was to create "host.sub.myservername.com" as an alias with email and then add root as a user (i.e., root@host.sub.myservername.com). But I was wary that because Virtualmin already sends email from that address without a subdomain or alias, it would screw something else up. I'm a bit wary still of doing this but as you've gotten me this far, I'll do this and report back with any problems.

Many, many thanks!

@c-prompt
Copy link
Author

c-prompt commented Apr 18, 2024

Bonus, To make your email pass DKIM also, just go to virtualmin and add "host.sub.myservername.com" as a virtual server or sub server, enable DKIM on it, and while you're at it, enable SPF and DMARC, so that all three of these features' DNS entries will be managed.

Nope... sadly, it appears that's not going to work. After doing this, I stopped receiving emails on all domains. According to the mail logs, it's attributable to an "unreasonable virtual_alias_maps map nesting" (whatever that means). For example, I sent a test mail from Gmail to one of my Virtualmin domains:

2024-04-18T12:02:48.177537-04:00 host postfix/smtpd[1733404]: 2B3FC67189: client=mail-lf1-f42.google.com[209.85.167.42]
2024-04-18T12:02:48.495427-04:00 host postfix/cleanup[1733410]: warning: 2B3FC67189: unreasonable virtual_alias_maps map nesting for my_email@domainname.com -- message not accepted, try again later
2024-04-18T12:02:48.627086-04:00 host postfix/smtpd[1733404]: disconnect from mail-lf1-f42.google.com[209.85.167.42] ehlo=2 starttls=1 mail=1 rcpt=1 bdat=0/1 quit=1 commands=6/7

Once I deleted the host subdomain, the error disappeared and email came through properly. Thus, if there isn't a way to add DKIM to host without creating a subdomain for host, perhaps I'll just stick with SPF and DMARC being OK and leave it at that.

@c-prompt c-prompt reopened this Apr 18, 2024
@chris001
Copy link

That error unreasonable virtual_alias_maps map nesting for my_email@domainname.com is something different. It's a infinite recursive text replacement of your email alias to an email address that's unreasonably long, because it's infinite. Solution: https://archive.virtualmin.com/node/8661

@c-prompt
Copy link
Author

Thanks. The challenge is that the error occurs when adding host as a subdomain. (I tried it on both servers and it happened both times.) So there's something Virtualmin doesn't like about adding host as a subdomain.

@c-prompt
Copy link
Author

Saw many errors like this in my mail.logs which I assume is related to what happens when adding host as a subdomain:

2024-04-18T06:50:10.036128-04:00 host postfix/trivial-rewrite[1657099]: warning: do not list domain host.myservername.com in BOTH mydestination and virtual_alias_domains

@chris001
Copy link

Virtualmin and Plesk might sometimes add the server's domain to both lists by default. It's redundant.
I'd comment out that line from the postfix config file's virtual_alias_domains, reload postfix service, and check if the warning stops occurring in the mail log.

@c-prompt
Copy link
Author

Fair enough. Although Perhaps a feature request would be for Virtualmin to check postfix before adding. I'd imagine it shouldn't be hard to do with a regex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants