Bug in selective service

[tomav]

Bug found by @ep0ch2 in #250

Many thanks for this, the following settings work:

• DISABLE_AMAVIS=0
• DISABLE_SPAMASSASSIN=1
• DISABLE_CLAMAV=1

however with disable_amavis=1 i can't send/receive emails. so perhaps some extra config changes are needed,

@KyleOndy: I'm opening this issue for reminder.
Will you be able to work on it? May I help?

Thank you.

[KyleOndy]

I went on a yak-shaving expedition attempting to expand the testing suite to catch issues like this. Let me look back into it.

[KyleOndy]

These next few weeks I am going to have very little or no availability. Anyone is welcome to take a look at this and submit a PR.

[tomav]

@KyleOndy, any update on these 2 issues #256 and #324 ? Thanks.

[kachkaev]

Great that this issue has been reported! I struggled to figure out what went wrong after setting DISABLE_AMAVIS=1 :–)

It seems reasonable to switch to an alternative service config as suggested in #297 while fixing this bug – it will require some config refactoring anyway.

IMHO not having a spam filter and an anti-virus has more pros than cons since it suggests a lightweight start with an option for extra nice features on demand. Otherwise, lots of users end up with a large chunk of their RAM swallowed by the container while they only use it as a simple ‘reverse-proxy’ for their corporate or private mail.

[KyleOndy]

I have not looked into this issue much as I have still been trying to figure out an acceptable way to remedy #324.

I may switch gears in the next week and at least push out a simple fix that will work for the time being.

[kachkaev]

I was able to turn off virus and spam checking in amavis while keeping DKIP and it had a dramatic impact on RAM usage:

(a peak before the break is an example how much memory each incoming message consumed!)

The solution was to create config/amavis.cf (thanks to #299) with the following content:

use strict;

@bypass_virus_checks_maps = (1);
@bypass_spam_checks_maps  = (1);

#------------ Do not modify anything below this line -------------
1;  # ensure a defined return

Now I am not sure that we need to have an option to remove AMAVIS at all because most users would probably want to keep DKIM at least. People can be just instructed to turn of the features that they don't want.

[KyleOndy]

That is a pretty big reduction in resources @kachkaev.

You bring up a good point regaring keeping AMAVIS to maintain DKIM. I agree that should be the recommendation for most users.

I am going to attempt to turn off the virus and spam checking on my personal server as I am moving to a smaller host. Once I get a clean implementation I'll open up a PR.

---

Also, what dashboard / munitioning system are you using @kachkaev? I like the looks of it.

[tomav]

Grafana probably.

[kachkaev]

@KyleOndy it's netdata – really great stuff!

[kachkaev]

One important thing to note before anyone else has started to encounter this. I don't know if this is related to turning off spam and virus filtering, but I've recently started to spot these pretty sad warnings in the logs from time to time:

mail postfix/qmgr[1038]: 93A1BBF5: from=<bounces+848413-8bd4-my.custom.address=example.com@sgmail.github.com>, size=9427, nrcpt=1 (queue active)
mail postfix/smtp[4308]: Trusted TLS connection established to gmail-smtp-in.l.google.com[173.194.222.26]:25: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
mail postfix/smtp[4308]: 93A1BBF5: host gmail-smtp-in.l.google.com[173.194.222.26] said: 421-4.7.0 [42.MY.IP.42      15] Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0  https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.0 review our Bulk Email Senders Guidelines. d185si8612556lfe.262 - gsmtp (in reply to end of DATA command)
mail postfix/smtp[4308]: connect to gmail-smtp-in.l.google.com[2a00:1450:4010:c07::1a]:25: Network is unreachable
mail postfix/smtp[4308]: Trusted TLS connection established to alt1.gmail-smtp-in.l.google.com[74.125.198.26]:25: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)
mail postfix/smtp[4308]: 93A1BBF5: to=<my.gmail.address@gmail.com>, relay=alt1.gmail-smtp-in.l.google.com[74.125.198.26]:25, delay=38374, delays=38372/0.01/1.7/0.53, dsn=4.7.0, status=deferred (host alt1.gmail-smtp-in.l.google.com[74.125.198.26] said: 421-4.7.0 [42.MY.IP.42      15] Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0  https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.0 review our Bulk Email Senders Guidelines. v47si9447174otv.121 - gsmtp (in reply to end of DATA command))

Key message (from gmail):

Our system has detected an unusual rate of 421-4.7.0 unsolicited mail originating from your IP address. To protect our 421-4.7.0 users from spam, mail sent from your IP address has been temporarily 421-4.7.0 rate limited. Please visit 421-4.7.0 https://support.google.com/mail/?p=UnsolicitedRateLimitError to 421 4.7.0 review our Bulk Email Senders Guidelines. d185si8612556lfe.262 - gsmtp (in reply to end of DATA command)

It turns out that some attempts to forward notifications from github, ebay and other services to gmail via my custom email server fail. I was doing this kind of forwarding for years without using fancy things like DKIM, Letsencrypt, DNS records, etc. and never heard of this issue before. A shallow search reveals that this a side-effect of google's recently updated spam policy. What is interesting is that real spam is either successfully discarded by zen.spamhaus.org on my server or is successfully swallowed by google and placed into the gmail's spam folder. My personal emails (hopefully) go well in both directions. Mail-testing services like http://mail-tester.com/ & http://mxtoolbox.com/ say that my settings a perfect. Neither IP address nor host name appear in any ban lists.

Not sure that this is something to be discussed in the thread or even in this repo, but I've still decided to share such a finding as this might save others some time. I'll be happy to join a discussion in another issue thread if this problem does not look to irrelevant to docker-mailserver.

[tomav]

Hi @KyleOndy, have you been able to fix the selective service startup?
Thank you.

[tomav]

I'm gonna work on this one.
DISABLE_AMAVIS will me removed to let DKIM work as expected.
DISABLE_CLAMAV will remain untouched.
DISABLE_SPAMASSASSIN does not exist in the code :-)
(I'll see if I can add it)

[tomav]

And I think everything should be ENABLE_ and not DISABLE_.