Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rspamd symbol HFILTER_HOSTNAME_UNKNOWN #3168

Closed
3 of 5 tasks
nightah opened this issue Nov 24, 2019 · 20 comments
Closed
3 of 5 tasks

rspamd symbol HFILTER_HOSTNAME_UNKNOWN #3168

nightah opened this issue Nov 24, 2019 · 20 comments

Comments

@nightah
Copy link
Contributor

nightah commented Nov 24, 2019

Prior to placing the issue, please check following: (fill out each checkbox with a X once done)

  • I understand that not following below instructions might result in immediate closing and deletion of my issue.
  • I have understood that answers are voluntary and community-driven, and not commercial support.
  • I have verified that my issue has not been already answered in the past. I also checked previous issues.

Description of the bug: What kind of issue have you exactly come across?

Regularly seeing the HFILTER_HOSTNAME_UNKNOWN rspamd symbol, since this has a very high rating almost all mail is rejected/quarantined.

This issue appears to be similar to #2014 although there's no SMTP proxy in my case.

Here's the Postfix log for two cases of offending mail:


24/11/2019, 06:44:10 | info | disconnect from unknown[192.174.88.224] ehlo=2 starttls=1 mail=1 rcpt=1 data=0/1 quit=1 commands=6/7
-- | -- | --
24/11/2019, 06:44:05 | info | 564E87BCE3: milter-reject: END-OF-MESSAGE from unknown[192.174.88.224]: 5.7.1 This message does not meet our delivery requirements; from=<bounces+user=example.com@mailer.humblebundle.com> to=<user@example.com> proto=ESMTP helo=<mta-88-224.sparkpostmail.com>
24/11/2019, 06:43:53 | info | 564E87BCE3: message-id=<AB.B5.08368.9EB89DD5@ad.mta2vrest.cc.prd.sparkpost>
24/11/2019, 06:43:52 | info | 564E87BCE3: client=unknown[192.174.88.224]
24/11/2019, 06:43:51 | info | Anonymous TLS connection established from unknown[192.174.88.224]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
24/11/2019, 06:43:50 | info | connect from unknown[192.174.88.224]
24/11/2019, 06:43:40 | info | WHITELISTED [192.174.88.224]:9718
24/11/2019, 06:43:40 | info | CONNECT from [192.174.88.224]:9718 to [172.20.0.6]:25
24/11/2019, 21:38:01 | info | disconnect from unknown[72.166.182.12] ehlo=2 starttls=1 mail=1 rcpt=1 data=0/1 quit=1 commands=6/7
-- | -- | --
24/11/2019, 21:38:01 | info | 31164108B49: milter-reject: END-OF-MESSAGE from unknown[72.166.182.12]: 5.7.1 This message does not meet our delivery requirements; from=<em-b58vs4ybzyr6xjaug9u70b7brtq5t5@m.email.sonyentertainmentnetwork.com> to=<user@example.com> proto=ESMTP helo=<nj5pebmpriv03.email.sonyentertainmentnetwork.com>
24/11/2019, 21:37:48 | info | 31164108B49: message-id=<b58vs4ybzyr6xjaug9u70b7brtq5t5@nj5pebmpriv03.email.sonyentertainmentnetwork.com>
-- | -- | --
24/11/2019, 21:37:48 | info | 31164108B49: client=unknown[72.166.182.12]
24/11/2019, 21:37:46 | info | Anonymous TLS connection established from unknown[72.166.182.12]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
24/11/2019, 21:37:45 | info | connect from unknown[72.166.182.12]
24/11/2019, 21:37:38 | info | addr 72.166.182.12 listed by domain list.dnswl.org as 127.0.15.0
24/11/2019, 21:37:35 | info | addr 72.166.182.12 listed by domain hostkarma.junkemailfilter.com as 127.0.1.1
24/11/2019, 21:37:35 | info | PASS NEW [72.166.182.12]:58752
24/11/2019, 21:37:32 | info | Look up 72.166.182.12 on whitelist, result 200 DUNNO
24/11/2019, 21:37:32 | info | CONNECT from [72.166.182.12]:58752 to [172.20.0.6]:25

Reproduction of said bug: How exactly do you reproduce the bug?

  1. Receive mail that gets tagged with symbol

I have tried or I do... (fill out each checkbox with a X if applicable)

  • In case of WebUI issue, I have tried clearing the browser cache and the issue persists.
  • I do run mailcow on a Synology, QNAP or any other sort of NAS.

System information

Further information (where applicable):

Question Answer
My operating system Arch Linux
Is Apparmor, SELinux or similar active? No
Virtualization technlogy (KVM, VMware, Xen, etc) n/a
Server/VM specifications (Memory, CPU Cores) 256GB Ram, 20 Cores
Docker Version (docker version) 19.03.5-ce
Docker-Compose Version (docker-compose version) 1.25.0
Reverse proxy (custom solution) nginx

Further notes:

  • Output of git diff origin/master, any other changes to the code? If so, please post them.
diff --git a/data/conf/postfix/mailcow_anonymize_headers.pcre_ b/data/conf/postfix/mailcow_anonymize_headers.pcre_
new file mode 100644
index 00000000..1d678ffa
--- /dev/null
+++ b/data/conf/postfix/mailcow_anonymize_headers.pcre_
@@ -0,0 +1,9 @@
+if /^\s*Received:.*Authenticated sender.*\(Postcow\)/
+/^\s*Received:.*Authenticated sender:(.+)/
+  REPLACE Received: from localhost (localhost [127.0.0.1]) (Authenticated sender:$1
+/^\s*User-Agent/        IGNORE
+/^\s*X-Enigmail/        IGNORE
+/^\s*X-Mailer/          IGNORE
+/^\s*X-Originating-IP/  IGNORE
+/^\s*X-Forward/         IGNORE
+endif
diff --git a/docker-compose.yml b/docker-compose.yml
index 3fd6e2e2..f38e985a 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -312,6 +312,9 @@ services:
         mailcow-network:
           aliases:
             - nginx
+        nerv_net:
+          aliases:
+            - mailcow

     acme-mailcow:
       depends_on:
@@ -492,6 +495,8 @@ networks:
       config:
         - subnet: ${IPV4_NETWORK:-172.22.1}.0/24
         - subnet: ${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
+  nerv_net:
+    external: true

 volumes:
   # Storage for email files
  • All third-party firewalls and custom iptables rules are unsupported. Please check the Docker docs about how to use Docker with your own ruleset. Nevertheless, iptabels output can help us to help you: iptables -L -vn, ip6tables -L -vn, iptables -L -vn -t nat and ip6tables -L -vn -t nat
Chain INPUT (policy ACCEPT 22572 packets, 5917K bytes)
 pkts bytes target     prot opt in     out     source               destination
22963 5949K MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 566K  903M MAILCOW    all  --  *      *       0.0.0.0/0            0.0.0.0/0
 570K  911M DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
 570K  911M DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
 133K  199M ACCEPT     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0
67653 3684K ACCEPT     all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-mailcow br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
14011 7342K ACCEPT     all  --  *      br-4c7763065d61  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-4c7763065d61  0.0.0.0/0            0.0.0.0/0
23064 2466K ACCEPT     all  --  br-4c7763065d61 !br-4c7763065d61  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-4c7763065d61 br-4c7763065d61  0.0.0.0/0            0.0.0.0/0
  52M   53G ACCEPT     all  --  *      br-b98aa365630e  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
1870K  343M DOCKER     all  --  *      br-b98aa365630e  0.0.0.0/0            0.0.0.0/0
  53M   94G ACCEPT     all  --  br-b98aa365630e !br-b98aa365630e  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-b98aa365630e br-b98aa365630e  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 17400 packets, 46M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (4 references)
 pkts bytes target     prot opt in     out     source               destination
    3   156 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.20          tcp dpt:6501
  257 13491 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.23          tcp dpt:80
  396 23532 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.11          tcp dpt:51413
  140  7318 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.22          tcp dpt:443
  278 14456 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.254         tcp dpt:53
23360 1558K ACCEPT     udp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.254         udp dpt:53
    0     0 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.24          tcp dpt:8989
    0     0 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.24          tcp dpt:4444
    0     0 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.24          tcp dpt:636
    0     0 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.24          tcp dpt:389
1160K  237M ACCEPT     udp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.26          udp dpt:4739
77142   64M ACCEPT     udp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.39          udp dpt:4739
 545K   34M ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.41          tcp dpt:443
 1054 44168 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.41          tcp dpt:80
    0     0 ACCEPT     udp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.9           udp dpt:10001
    1    64 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.9           tcp dpt:8880
    0     0 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.9           tcp dpt:8443
 5228  314K ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.9           tcp dpt:8080
    0     0 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.9           tcp dpt:6789
  869 48664 ACCEPT     udp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.9           udp dpt:3478
19606 1176K ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.18          tcp dpt:3306
    0     0 ACCEPT     tcp  --  !br-b98aa365630e br-b98aa365630e  0.0.0.0/0            172.18.0.12          tcp dpt:32400
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.3           tcp dpt:587
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.3           tcp dpt:465
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.3           tcp dpt:25
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.6           tcp dpt:8983
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.250         tcp dpt:12345
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.250         tcp dpt:4190
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.250         tcp dpt:995
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.250         tcp dpt:993
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.250         tcp dpt:143
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.250         tcp dpt:110
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.8           tcp dpt:3306
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.13          tcp dpt:9443
    0     0 ACCEPT     tcp  --  !br-mailcow br-mailcow  0.0.0.0/0            172.20.0.13          tcp dpt:9000

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
67653 3684K DOCKER-ISOLATION-STAGE-2  all  --  br-mailcow !br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
23064 2466K DOCKER-ISOLATION-STAGE-2  all  --  br-4c7763065d61 !br-4c7763065d61  0.0.0.0/0            0.0.0.0/0
  53M   94G DOCKER-ISOLATION-STAGE-2  all  --  br-b98aa365630e !br-b98aa365630e  0.0.0.0/0            0.0.0.0/0
 107M  148G RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (4 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      br-mailcow  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      br-4c7763065d61  0.0.0.0/0            0.0.0.0/0
    0     0 DROP       all  --  *      br-b98aa365630e  0.0.0.0/0            0.0.0.0/0
  53M   94G RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
 107M  148G RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MAILCOW (2 references)
 pkts bytes target     prot opt in     out     source               destination
Chain INPUT (policy ACCEPT 375 packets, 25384 bytes)
 pkts bytes target     prot opt in     out     source               destination
  390 26224 MAILCOW    all      *      *       ::/0                 ::/0

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-USER  all      *      *       ::/0                 ::/0
    0     0 MAILCOW    all      *      *       ::/0                 ::/0
    0     0 DOCKER     all      *      br-mailcow  ::/0                 ::/0
    0     0 ACCEPT     all      *      br-mailcow  ::/0                 ::/0                 ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     all      br-mailcow !br-mailcow  ::/0                 ::/0
    0     0 ACCEPT     all      br-mailcow br-mailcow  ::/0                 ::/0
    0     0 DOCKER-ISOLATION-STAGE-1  all      *      *       ::/0                 ::/0

Chain OUTPUT (policy ACCEPT 63 packets, 8206 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::11  tcp dpt:9000
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::11  tcp dpt:9443
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::5  tcp dpt:25
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::5  tcp dpt:465
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::5  tcp dpt:587
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:110
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:143
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:4190
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:993
    0     0 ACCEPT     tcp      !br-mailcow br-mailcow  ::/0                 fd4d:6169:6c63:6f77::9  tcp dpt:995

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-ISOLATION-STAGE-2  all      br-mailcow !br-mailcow  ::/0                 ::/0
    0     0 RETURN     all      *      *       ::/0                 ::/0

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all      *      br-mailcow  ::/0                 ::/0
    0     0 RETURN     all      *      *       ::/0                 ::/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all      *      *       ::/0                 ::/0

Chain MAILCOW (2 references)
 pkts bytes target     prot opt in     out     source               destination
Chain PREROUTING (policy ACCEPT 2720 packets, 376K bytes)
 pkts bytes target     prot opt in     out     source               destination
 639K   40M DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 122 packets, 9832 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 274 packets, 18080 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 2045 packets, 125K bytes)
 pkts bytes target     prot opt in     out     source               destination
  110  7903 MASQUERADE  all  --  *      !br-mailcow  172.20.0.0/24        0.0.0.0/0
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0
  624 38119 MASQUERADE  all  --  *      !br-4c7763065d61  172.19.0.0/16        0.0.0.0/0
 192K   17M MASQUERADE  all  --  *      !br-b98aa365630e  172.18.0.0/16        0.0.0.0/0
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.20          172.18.0.20          tcp dpt:6501
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.23          172.18.0.23          tcp dpt:80
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.11          172.18.0.11          tcp dpt:51413
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.22          172.18.0.22          tcp dpt:443
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.254         172.18.0.254         tcp dpt:53
    0     0 MASQUERADE  udp  --  *      *       172.18.0.254         172.18.0.254         udp dpt:53
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.24          172.18.0.24          tcp dpt:8989
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.24          172.18.0.24          tcp dpt:4444
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.24          172.18.0.24          tcp dpt:636
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.24          172.18.0.24          tcp dpt:389
    0     0 MASQUERADE  udp  --  *      *       172.18.0.26          172.18.0.26          udp dpt:4739
    0     0 MASQUERADE  udp  --  *      *       172.18.0.39          172.18.0.39          udp dpt:4739
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.41          172.18.0.41          tcp dpt:443
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.41          172.18.0.41          tcp dpt:80
    0     0 MASQUERADE  udp  --  *      *       172.18.0.9           172.18.0.9           udp dpt:10001
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.9           172.18.0.9           tcp dpt:8880
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.9           172.18.0.9           tcp dpt:8443
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.9           172.18.0.9           tcp dpt:8080
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.9           172.18.0.9           tcp dpt:6789
    0     0 MASQUERADE  udp  --  *      *       172.18.0.9           172.18.0.9           udp dpt:3478
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.18          172.18.0.18          tcp dpt:3306
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.12          172.18.0.12          tcp dpt:32400
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.3           172.20.0.3           tcp dpt:587
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.3           172.20.0.3           tcp dpt:465
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.3           172.20.0.3           tcp dpt:25
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.6           172.20.0.6           tcp dpt:8983
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.250         172.20.0.250         tcp dpt:12345
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.250         172.20.0.250         tcp dpt:4190
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.250         172.20.0.250         tcp dpt:995
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.250         172.20.0.250         tcp dpt:993
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.250         172.20.0.250         tcp dpt:143
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.250         172.20.0.250         tcp dpt:110
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.8           172.20.0.8           tcp dpt:3306
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.13          172.20.0.13          tcp dpt:9443
    0     0 MASQUERADE  tcp  --  *      *       172.20.0.13          172.20.0.13          tcp dpt:9000

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  br-mailcow *       0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0
 7976  479K RETURN     all  --  br-4c7763065d61 *       0.0.0.0/0            0.0.0.0/0
  217 13020 RETURN     all  --  br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0
    3   156 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:6667 to:172.18.0.20:6501
  257 13491 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            10.10.10.200         tcp dpt:80 to:172.18.0.23:80
  396 23532 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:51413 to:172.18.0.11:51413
  140  7318 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            10.10.10.200         tcp dpt:443 to:172.18.0.22:443
  278 14456 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53 to:172.18.0.254:53
23366 1558K DNAT       udp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 to:172.18.0.254:53
    0     0 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8989 to:172.18.0.24:8989
    0     0 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:4444 to:172.18.0.24:4444
    0     0 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:636 to:172.18.0.24:636
    0     0 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:389 to:172.18.0.24:389
   10  3604 DNAT       udp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            udp dpt:4739 to:172.18.0.26:4739
    3   900 DNAT       udp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            udp dpt:4738 to:172.18.0.39:4739
 545K   34M DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            10.10.10.1           tcp dpt:443 to:172.18.0.41:443
 1059 44368 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            10.10.10.1           tcp dpt:80 to:172.18.0.41:80
    0     0 DNAT       udp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            udp dpt:10001 to:172.18.0.9:10001
    1    64 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8880 to:172.18.0.9:8880
    0     0 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8443 to:172.18.0.9:8443
 5237  314K DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8080 to:172.18.0.9:8080
    0     0 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:6789 to:172.18.0.9:6789
  869 48664 DNAT       udp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            udp dpt:3478 to:172.18.0.9:3478
19606 1176K DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3306 to:172.18.0.18:3306
    0     0 DNAT       tcp  --  !br-b98aa365630e *       0.0.0.0/0            0.0.0.0/0            tcp dpt:32400 to:172.18.0.12:32400
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:587 to:172.20.0.3:587
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:465 to:172.20.0.3:465
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:25 to:172.20.0.3:25
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:18983 to:172.20.0.6:8983
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:19991 to:172.20.0.250:12345
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:4190 to:172.20.0.250:4190
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:995 to:172.20.0.250:995
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:993 to:172.20.0.250:993
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:143 to:172.20.0.250:143
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:110 to:172.20.0.250:110
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            127.0.0.1            tcp dpt:13306 to:172.20.0.8:3306
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9443 to:172.20.0.13:9443
    0     0 DNAT       tcp  --  !br-mailcow *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9000 to:172.20.0.13:9000
Chain PREROUTING (policy ACCEPT 137 packets, 12790 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all      *      *       ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     all      *      *       ::/0                !::1                  ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 MASQUERADE  all      *      br-mailcow  ::/0                 ::/0                 ADDRTYPE match dst-type LOCAL
    0     0 MASQUERADE  all      *      !br-mailcow  fd4d:6169:6c63:6f77::/64  ::/0
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::11  fd4d:6169:6c63:6f77::11  tcp dpt:9000
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::11  fd4d:6169:6c63:6f77::11  tcp dpt:9443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::5  fd4d:6169:6c63:6f77::5  tcp dpt:25
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::5  fd4d:6169:6c63:6f77::5  tcp dpt:465
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::5  fd4d:6169:6c63:6f77::5  tcp dpt:587
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::9  fd4d:6169:6c63:6f77::9  tcp dpt:110
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::9  fd4d:6169:6c63:6f77::9  tcp dpt:143
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::9  fd4d:6169:6c63:6f77::9  tcp dpt:4190
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::9  fd4d:6169:6c63:6f77::9  tcp dpt:993
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::9  fd4d:6169:6c63:6f77::9  tcp dpt:995
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::4  fd4d:6169:6c63:6f77::4  tcp dpt:9000
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::4  fd4d:6169:6c63:6f77::4  tcp dpt:9443
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::8  fd4d:6169:6c63:6f77::8  tcp dpt:25
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::8  fd4d:6169:6c63:6f77::8  tcp dpt:465
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::8  fd4d:6169:6c63:6f77::8  tcp dpt:587
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::2  fd4d:6169:6c63:6f77::2  tcp dpt:110
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::2  fd4d:6169:6c63:6f77::2  tcp dpt:143
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::2  fd4d:6169:6c63:6f77::2  tcp dpt:4190
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::2  fd4d:6169:6c63:6f77::2  tcp dpt:993
    0     0 MASQUERADE  tcp      *      *       fd4d:6169:6c63:6f77::2  fd4d:6169:6c63:6f77::2  tcp dpt:995

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all      br-mailcow *       ::/0                 ::/0
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:9000 to:[fd4d:6169:6c63:6f77::11]:9000
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:9443 to:[fd4d:6169:6c63:6f77::11]:9443
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:25 to:[fd4d:6169:6c63:6f77::5]:25
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:465 to:[fd4d:6169:6c63:6f77::5]:465
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:587 to:[fd4d:6169:6c63:6f77::5]:587
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:110 to:[fd4d:6169:6c63:6f77::9]:110
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:143 to:[fd4d:6169:6c63:6f77::9]:143
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:4190 to:[fd4d:6169:6c63:6f77::9]:4190
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:993 to:[fd4d:6169:6c63:6f77::9]:993
    0     0 DNAT       tcp      !br-mailcow *       ::/0                 ::/0                 tcp dpt:995 to:[fd4d:6169:6c63:6f77::9]:995
  • Check docker exec -it $(docker ps -qf name=acme-mailcow) dig +short stackoverflow.com @172.22.1.254 (set the IP accordingly, if you changed the internal mailcow network) and docker exec -it $(docker ps -qf name=acme-mailcow) dig +short stackoverflow.com @1.1.1.1 - output? Timeout?

Both return:

151.101.65.69
151.101.1.69
151.101.129.69
151.101.193.69
@andryyy
Copy link
Contributor

andryyy commented Nov 24, 2019

We removed the PTR check from Postfix and moved it to Rspamd.

@andryyy
Copy link
Contributor

andryyy commented Nov 24, 2019

It is a bit strange you see from unknown[192.174.88.224] as this IP has a valid PTR.

I think your unbound container is not working correctly. It should not have been flagged.

Anything in your unbound logs? Can you increase verbosity? I recommend you decrease the symbol score on your system to something lower OR reimplement the ptr check to Postfix while also monitoring unbound.

Verbosity unbound-mailcow:

data/conf/unbound/unbound.conf => verbosity: 2 and docker-compose restart unbound-mailcow

@nightah
Copy link
Contributor Author

nightah commented Nov 24, 2019

I noticed when performing a nslookup from within the Unbound containers on both hosts that they correctly resolved the appropriate PTR.

I’ll increase the log verbosity and continue to monitor for now. It’s not every single mail that gets caught but a significant amount seems to be.

@andryyy
Copy link
Contributor

andryyy commented Nov 24, 2019

If you use nslookup inside unbound, you need to query 127.0.0.1. With dig it is dig -x your.ip.add.ress @127.0.0.1. If you don't query against 127.0.0.1, you are querying against your hosts DNS.

Is it happening when the host is busy? You should reduce the symbols score in Rspamd UI while testing.

I think you only now found out about this issue, because mails pop up in your quaratine and Rspamd log. They were probably rejected before, but you only now noticed.

@nightah
Copy link
Contributor Author

nightah commented Nov 24, 2019

dig isn't installed inside the unbound container, you can just perform a nslookup <ipaddress> <dnsserver> to query against the local DNS, this is what I did both times earlier.

Also that could be the case but it seems unlikely, nobody ever reached out to suggest that mail hadn't been responded to.

@andryyy
Copy link
Contributor

andryyy commented Nov 24, 2019

Use dig in acme-mailcow and dig against @Unbound

@nightah
Copy link
Contributor Author

nightah commented Nov 24, 2019

I'm just simulating some mail in to see if I can find a failure with the verbosity increased on Unbound now.

What's the best place to debug this, I'm around on IRC if that helps.

@andryyy
Copy link
Contributor

andryyy commented Nov 24, 2019

I don't think that's a general issue. Join the Telegram group and post some logs. But I won't do anything today anymore. :) I might be around tomorrow evening.

@nightah
Copy link
Contributor Author

nightah commented Nov 25, 2019

Upon further investigation it looks like it's the very first mail that's received from mailcow after a reboot or restarting the full stack.

The first lookup takes a little longer as it resolves each NS/TLD chain. Can we perhaps force a lookup upon startup for Unbound?

@andryyy
Copy link
Contributor

andryyy commented Nov 25, 2019

We can do that. :)

@nightah
Copy link
Contributor Author

nightah commented Dec 2, 2019

So for anybody that is experiencing similar issues, I had mailcow configured as per default with ipv6 enabled despite not having an ipv6 enabled network.

That was causing issues with Unbound, specifically PTR lookups were taking much longer than they should and would often result in hosts with a valid PTR coming up as unknown and being marked with the HFILTER_HOSTNAME_UNKNOWN symbol.

I've submitted a PR for the docs mailcow/mailcow-dockerized-docs#182 which recommends disabling IPv6 if your network is not capable, this has prevented the issue in my scenario.

@nightah nightah closed this as completed Dec 2, 2019
@mkuron
Copy link
Member

mkuron commented Dec 2, 2019

Perhaps we can make the fallback to IPv4 happen faster? Is there such a config option in unbound?

@mkuron mkuron reopened this Dec 2, 2019
@nightah
Copy link
Contributor Author

nightah commented Dec 2, 2019

I couldn't find such a config, the only way I was able to prevent it from occurring without disabling IPv6 on the whole stack was to disable IPv6 for Unbound specifically do-ip6: no here:

@nightah
Copy link
Contributor Author

nightah commented Dec 2, 2019

@andryyy while not a big deal, it's probably still worthwhile changing the value above, whether that's done based on the watchdog check or by specifying in the documentation I'll let you decide.

The reason being, if you only disable ipv6 in the mailcow network Unbound will still be making attempts to ipv6 upstream and will silently error and fail, disabling it in Unbound might actually make more sense to prevent this all together.

@mkuron mkuron closed this as completed Dec 3, 2019
@warnerbryce
Copy link

I have a VPS server running MailCow with IPv6 networking working and i still have issues of HFILTER_HOSTNAME_UNKNOWN....

@andryyy
Copy link
Contributor

andryyy commented Jan 3, 2020

Why issues? It is a symbol, that triggers with missing/wrong PTR records. :) Have you validated, that your sender has a valid PTR?

@warnerbryce
Copy link

warnerbryce commented Jan 3, 2020

Yes, the log indicate an IPv4 address that match with the MX server of the sender...
EDIT : maybe you are right The PTR gave smtpout.banque-accord.com but this FQDN has no A record in DNS servers... So finally it's not a bad thing.

@andryyy
Copy link
Contributor

andryyy commented Jan 3, 2020

You can whitelist the sender in mailcow UI and inform the sender. :)

@warnerbryce
Copy link

That nearly what i did.
I just inform the sender about this issue, but i didn't whitelist it in MailcowUI.
I don't care if i don't have their emails, they just have to get a better automated-email service.
Thank you for the tip @andryyy, i was debugging well, but didn't investigate enough :-)

@andryyy
Copy link
Contributor

andryyy commented Jan 3, 2020

You are welcome. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants