imap3 port not defined in Debian Stretch

[milo]

Environment:

• Fail2Ban version (including any possible distribution suffixes): 0.9.6-2
• OS, including release name/version: Debian Stretch 9.2
• Fail2Ban installed via OS/distribution mechanisms
• I have not applied any additional foreign patches to the codebase
• Some customizations were done to the configuration (provide details below is so)

The issue:

I had defined a webmail spam jail:

[webmail-spam]

enabled  = true
filter   = webmail-spam
logpath  = /var/log/mail.err
port     = http,https,smtp,ssmtp,submission,imap2,imap3,imaps,pop3,pop3s
maxretry = 3
findtime = 180
bantime  = 28800
action   = %(action_mwl)s

Everything seems to work OK. I received an email with subject:

[Fail2Ban] webmail-spam: banned 41.190.3.6 from epsilon

But today I found, that even the fail2ban-client prints that IP is blocked, iptables rules were missing. In log I found:

2017-10-26 08:04:02,736 fail2ban.actions        [1636]: NOTICE  [webmail-spam] Ban 41.190.3.6
2017-10-26 08:04:02,843 fail2ban.action         [1636]: ERROR   iptables -w -n -L INPUT | grep -q 'f2b-webmail-spam[ \t]' -- stdout: b''
2017-10-26 08:04:02,843 fail2ban.action         [1636]: ERROR   iptables -w -n -L INPUT | grep -q 'f2b-webmail-spam[ \t]' -- stderr: b''
2017-10-26 08:04:02,843 fail2ban.action         [1636]: ERROR   iptables -w -n -L INPUT | grep -q 'f2b-webmail-spam[ \t]' -- returned 1
2017-10-26 08:04:02,843 fail2ban.CommandAction  [1636]: ERROR   Invariant check failed. Trying to restore a sane environment
2017-10-26 08:04:03,055 fail2ban.action         [1636]: ERROR   iptables -w -N f2b-webmail-spam
iptables -w -A f2b-webmail-spam -j RETURN
iptables -w -I INPUT -p tcp -m multiport --dports http,https,smtp,ssmtp,submission,imap2,imap3,imaps,pop3,pop3s -j f2b-webmail-spam -- stdout: b''
2017-10-26 08:04:03,055 fail2ban.action         [1636]: ERROR   iptables -w -N f2b-webmail-spam
iptables -w -A f2b-webmail-spam -j RETURN
iptables -w -I INPUT -p tcp -m multiport --dports http,https,smtp,ssmtp,submission,imap2,imap3,imaps,pop3,pop3s -j f2b-webmail-spam -- stderr: b"iptables v1.6.0: invalid port/service `imap3' specified\nTry `iptables -h' or 'iptables --help' for more information.\n"
2017-10-26 08:04:03,055 fail2ban.action         [1636]: ERROR   iptables -w -N f2b-webmail-spam
iptables -w -A f2b-webmail-spam -j RETURN
iptables -w -I INPUT -p tcp -m multiport --dports http,https,smtp,ssmtp,submission,imap2,imap3,imaps,pop3,pop3s -j f2b-webmail-spam -- returned 2
2017-10-26 08:04:03,055 fail2ban.actions        [1636]: ERROR   Failed to execute ban jail 'webmail-spam' action 'iptables-multiport' info 'CallingMap({'time': 1508997842.7366405, 'ipmatches': <function Actions.__checkBan.<locals>.<lambda> at 0x7f7880514598>, 'ipjailfailures': <function Actions.__checkBan.<locals>.<lambda> at 0x7f78806e6598>, 'matches': 'Oct 26 08:01:24 epsilon squirrelmail: Possible outgoing spam: by username (example.com) at 41.190.3.6 on 10/26/2017 06:01:24: Total 41 recipients... : Error starting action

Similar errors I got during fail2ban restart.

The solution for me was to remove imap3 from the jail definition, because it is missing in /etc/services after upgrade to Stretch.

Would be great if fail2ban refuse to start in such cases.

[sebres]

imap3 port not defined in Debian Stretch

I found no references about this neither in debian stretch changelog, nor somewhere else.
Also my /etc/services on stretch-machines has still this port entry...

$ grep VERSION /etc/os-release && grep imap3 /etc/services
VERSION_ID="9"
VERSION="9 (stretch)"
imap3           220/tcp                         # Interactive Mail Access
imap3           220/udp                         # Protocol v3

@yarikoptic, @fail2ban/maintainers What do you think about this?

Would be great if fail2ban refuse to start in such cases.

This is the matter of opinion: the one wants just to see a warning (and nevertheless to start fail2ban to protect another jails), another wants prevent the start completely.

But ATM I don't understood why this error let start fail2ban (even exactly with version 0.9.x)...
Because IMHO it could affect v.0.10 only...
For this purposes we have firstly introduced a logic "skip wrong confugured jails" and a new option -t (resp. --test), to avoid it, see #1619.
You can for example add this to the start parameters into fail2ban.service resp. init.d script to stop the whole starting process in some error case.
But for exactly your case, this will do nothing, because the fail2ban v.0.10 starts the actions on demand (because of conditional parameters like IPv6, etc., see #1742 for details).
But as already said, this affects version >= 0.10 only...

[yarikoptic]

heh heh ... thanks for the report

• apparently I had the same issue reported in Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867374 (and failed to react :-()
• apparently imap3 isn't defined in the standard /etc/services on debian, as of current netbase 5.4. I wonder if it should -- according to https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol#IMAP3 it is an old rarely (if ever) used. So may be we should just remove imap3 here (and/or replace with the port number)?

[sebres]

So may be we should just remove imap3 here (and/or replace with the port number)?

Well, as I understand, the mentioned webmail-spam is anyway custom jail/filter.
But I'll say - let remove it everywhere in our configs (I don't see that this port will be used per default)...

[sebres]

Fixed in #1812

[milo]

Hi! Have no idea why I didn't received notification for this. So... Thank you!

[sebres]

You're welcome.

[aziouk]

I was hit by this. Seems debian 9 still ships with this.

root@888:/etc/fail2ban#  grep -rnw '.' -e "imap3"
./jail.conf:610:port     = smtp,465,submission,imap3,imaps,pop3,pop3s
./jail.conf:617:port     = smtp,465,submission,imap3,imaps,pop3,pop3s
./jail.conf:627:port   = imap3,imaps,pop3,pop3s
./jail.conf:634:port = smtp,465,submission,imap2,imap3,imaps,pop3,pop3s,http,https,socks
./jail.conf:640:port   = imap3,imaps
./jail.conf:647:port   = imap3,imaps
root@scw-d06740:/etc/fail2ban# cat /etc/issue
Debian GNU/Linux 9 \n \l

root@888:/etc/fail2ban# apt-cache show fail2ban
Package: fail2ban
Version: 0.9.6-2
Installed-Size: 1290
Maintainer: Yaroslav Halchenko <debian@onerussian.com>
Architecture: all
Depends: python3:any (>= 3.3.2-2~), init-system-helpers (>= 1.18~), lsb-base (>= 2.0-7)
Recommends: python, iptables, whois, python3-pyinotify, python3-systemd
Suggests: mailx, system-log-daemon, monit
Description-en: ban hosts that cause multiple authentication errors
 Fail2ban monitors log files (e.g. /var/log/auth.log,
 /var/log/apache/access.log) and temporarily or persistently bans
 failure-prone addresses by updating existing firewall rules.  Fail2ban
 allows easy specification of different actions to be taken such as to ban
 an IP using iptables or hostsdeny rules, or simply to send a notification
 email.
 .
 By default, it comes with filter expressions for various services
 (sshd, apache, qmail, proftpd, sasl etc.) but configuration can be
 easily extended for monitoring any other text file.  All filters and
 actions are given in the config files, thus fail2ban can be adopted
 to be used with a variety of files and firewalls.  Following recommends
 are listed:
 .
  - iptables -- default installation uses iptables for banning. You most
    probably need it
  - whois -- used by a number of *mail-whois* actions to send notification
    emails with whois information about attacker hosts. Unless you will use
    those you don't need whois
  - python3-pyinotify -- unless you monitor services logs via systemd, you
    need pyinotify for efficient monitoring for log files changes
Description-md5: 95d3181b6813e8ad44755ae80b3619f9
Homepage: http://www.fail2ban.org
Tag: admin::automation, admin::logging, admin::monitoring,
 implemented-in::python, interface::commandline, interface::daemon,
 network::firewall, protocol::ip, role::program,
 security::authentication, security::firewall, security::ids,
 security::log-analyzer, use::login, use::monitor,
 works-with-format::plaintext, works-with::logfile, works-with::text
Section: net
Priority: optional
Filename: pool/main/f/fail2ban/fail2ban_0.9.6-2_all.deb
Size: 287540
MD5sum: 296e18a9b089bfefaef25489cb22fd25
SHA256: e68bb7afbd68c53741630c834ab78f093bfc393e88f40b37dc5904f773a371dd

Hope im not digging this up unnecessarily, but in any case hope you appreciate it. I issued an upgrade, but didn't see any updates for it in the repo.

Cause I know you may ask;

stretch-repo's used

root@scw-d06740:/etc/apt# cat sources.list
deb http://ftp.fr.debian.org/debian/ stretch main contrib non-free
deb-src http://ftp.fr.debian.org/debian/ stretch main contrib non-free

# Security
deb http://security.debian.org/ stretch/updates main contrib non-free
deb-src http://security.debian.org/ stretch/updates main contrib non-free

# Backports
deb http://ftp.debian.org/debian/ stretch-backports main contrib non-free
root@scw-d06740:/etc/apt#

Evidently I do hope that changing the imap3 entries to 'imap' will fix it ;-)

Best,
Adam

[aziouk]

ack, sadly still get

2018-08-16 18:03:36,280 fail2ban.actions        [5489]: NOTICE  [sshd] Ban someip.hoes.here.censored
2018-08-16 18:03:36,281 fail2ban.action         [5489]: DEBUG   iptables -w -n -L INPUT | grep -q 'f2b-sshd[ \t]'
2018-08-16 18:03:36,383 fail2ban.action         [5489]: ERROR   iptables -w -N f2b-postfix-sasl
iptables -w -A f2b-postfix-sasl -j RETURN
iptables -w -I INPUT -p tcp --dport smtp,465,submission,imap,imaps,pop3,pop3s -j f2b-postfix-sasl -- stdout: b''
2018-08-16 18:03:36,384 fail2ban.action         [5489]: ERROR   iptables -w -N f2b-postfix-sasl
iptables -w -A f2b-postfix-sasl -j RETURN
iptables -w -I INPUT -p tcp --dport smtp,465,submission,imap,imaps,pop3,pop3s -j f2b-postfix-sasl -- stderr: b"iptables v1.6.0: invalid                                                                                                      port/service `smtp,465,submission,imap,imaps,pop3,pop3s' specified\nTry `iptables -h' or 'iptables --help' for more information.\n"
2018-08-16 18:03:36,385 fail2ban.action         [5489]: ERROR   iptables -w -N f2b-postfix-sasl
iptables -w -A f2b-postfix-sasl -j RETURN
iptables -w -I INPUT -p tcp --dport smtp,465,submission,imap,imaps,pop3,pop3s -j f2b-postfix-sasl -- returned 2
2018-08-16 18:03:36,385 fail2ban.actions        [5489]: ERROR   Failed to execute ban jail 'postfix-sasl' action 'iptables' info 'Callin                                                                                                     gMap({'ip': 'some.censored.ip.here', 'time': 1534442615.751562, 'ipfailures': <function Actions.__checkBan.<locals>.<lambda> at 0x7f87cfbcb7b8                                                                                                     >, 'ipmatches': <function Actions.__checkBan.<locals>.<lambda> at 0x7f87cfbcb840>, 'failures': 145, 'matches': 'Aug 16 16:22:11 mail pos               

root@scw-d06740:/etc/fail2ban#  grep -rnw '.' -e "imap3"
root@888:/etc/fail2ban#

It is interesting to note that the SSHD iptable chain is dropping people passing maxretry=5 just fine. Wheras SASL-postfix filter.d is not. I think it is regarding the issue you raise earlier, though replacing the imap3 with imap does not seem to have fixed it. Can you clarify on the fix I should apply ?

Cheers :)

Best wishes,
Adam

[sebres]

Your excerpt is cut off (stderr: b"iptables v1.6.0: invalid , one does not see what is invalid)
Possibly some iptables-entries still present (after stop/restart), if you stop fail2ban and see some iptables chains of fail2ban, please remove it before you try to start next time.

As regards "but didn't see any updates for it in the repo.", - as #1812 said - it is fixed for v.0.10 only.
Do not have any influence on the @fail2ban/maintainers (@fail2ban/debian), so either use another repo for newest fail2ban or wait for updates (or even fix it self).

[aziouk]

Thanks for writing and clarifying. I didn't see that it was fixed for 0.10 only.

so, I thought I'd just keep on removing from the jail.conf file the ports used that iptbles 1.6 wasnt happy with. In my case I only really wanted SMTP anyway, I'll add the rest later. Someone ip range was being particularly bothersome today.

Apologies invalid port/service.

iptables -w -I INPUT -p tcp --dport smtp,imap,imaps,pop3,pop3s -j f2b-postfix-sasl -- stderr: b"iptables v1.6.0: invalid port/service `smtp,imap,imaps,pop3,pop3s' specified\nTry `iptables -h' or 'iptables --help' for more information.\n"
2018-08-16 18:18:57,712 fail2ban.action         [6092]: ERROR   iptables -w -N f2b-postfix-sasl
iptables -w -A f2b-postfix-sasl -j RETURN

simply adding

[postfix-sasl]

port     = smtp
--
# running postfix since it would provide the same log lines at the
# "warn" level but overall at the smaller filesize.
logpath  = %(postfix_log)s
backend  = %(postfix_backend)s

as opposed to http,https,smtp,ssmtp,submission,imap2,imap,imaps,pop3,pop3s. Seemed to fix it. Removing just imap3 didn't. If that's useful great. At least my reject in the postfix-sasl chain now appear since i shortened the line i refer.

If this helps, great.

Best wishes,
Adam

[sebres]

Removing just imap3 didn't.

Could you please have a look at which other ports/aliases you still miss in your /etc/services?
Have you changed it (or it is still original from the distribution)?

[aziouk]

Hey sebres, apologies, I didn't see your message as my old work email was still getting notifications.

If I understand your question correct. I believe it should be the default /etc/services file.

root@mail:~# stat /etc/services
  File: /etc/services
  Size: 19183           Blocks: 40         IO Block: 4096   regular file
Device: 2b00h/11008d    Inode: 1311298     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-08-19 21:08:33.224200409 +0000
Modify: 2016-12-26 01:56:39.000000000 +0000
Change: 2017-06-29 10:18:23.950225000 +0000

I'm not sure but I guess this is approximately what you wanted to know was?

root@mail:~# cat /etc/services  | grep imap
imap2           143/tcp         imap            # Interim Mail Access P 2 and 4
imaps           993/tcp                         # IMAP over SSL
root@mail:~# cat /etc/services  | grep -i imap
imap2           143/tcp         imap            # Interim Mail Access P 2 and 4
imaps           993/tcp                         # IMAP over SSL
root@mail:~# cat /etc/services  | grep -i pop
pop3            110/tcp         pop-3           # POP version 3
pop3s           995/tcp                         # POP-3 over SSL
kpop            1109/tcp                        # Pop with Kerberos
poppassd        106/tcp                         # Eudora
poppassd        106/udp

I think that it is something to do with more than just the imap alias was my point, and I suspect iptables has an issue with the use of the others, as to why that is, you seem to know more on this situation than me.

Thanks for the software though, I'm pleased to anounce my f2b-posfix-sasl chain is now working as expected. Simply just using the modifications I suggested above. I'm just guessing one of the other settings (other than SMTP) just wasn't appreciated by iptables, and perhaps that is something to do with the /etc/services file I suppose is your point.

Anyway, sorry for delay in responding and hope this helps some, if not, oh well :-)

Best,
adam