fail2ban not releasing ban after expiration #34

pittbull · 2021-04-05T09:20:08Z

I seem to have an issue where fail2ban reactives a previous ban on an ip when the same ip tries to access the site after ban expires.

Environment:

traefik 2.4.8
fail2ban 0.6.0

middelwares.toml
  [http.middlewares.my-fail2ban]
    [http.middlewares.my-fail2ban.plugin]
      [http.middlewares.my-fail2ban.plugin.fail2ban]
        [http.middlewares.my-fail2ban.plugin.fail2ban.blacklist]
          ip = [""]
        [http.middlewares.my-fail2ban.plugin.fail2ban.rules]
          action = ""
          actionAbuseipdb = ""
          backend = ""
          banaction = ""
          banactionAllports = ""
          bantime = "1m"
          chain = ""
          destemail = ""
          enabled = true
          fail2banAgent = ""
          filter = ""
          findtime = "1m"
          ignorecommand = ""
          logencoding = "UTF-8"
          maxretry = 4
          mode = ""
          mta = ""
          ports = "0:8000"
          protocol = ""
          sender = ""
          urlregexp = ""
          usedns = ""
        [http.middlewares.my-fail2ban.plugin.fail2ban.whitelist]
          ip = ["::1", "127.0.0.1", "192.168.1.0/24", "172.16.1.0/16"]

static config:

      - --pilot.token=<redacted>
      - --experimental.plugins.fail2ban.modulename=github.com/tommoulard/fail2ban
      - --experimental.plugins.fail2ban.version=v0.6.0

service config:
- "traefik.http.routers.authelia-rtr.middlewares=my-fail2ban@file,chain-no-auth@file"

Traefik log after restart:

Fail2Ban_config: 2021/04/05 08:57:45 restricted.go:51: Whitelisted: '::1/128'
Fail2Ban_config: 2021/04/05 08:57:45 restricted.go:51: Whitelisted: '127.0.0.1/32'
Fail2Ban_config: 2021/04/05 08:57:45 restricted.go:51: Whitelisted: '192.168.1.0/24'
Fail2Ban_config: 2021/04/05 08:57:45 restricted.go:51: Whitelisted: '172.16.0.0/16'
Fail2Ban_config: 2021/04/05 08:57:45 restricted.go:51: Bantime: 1m0s
Fail2Ban_config: 2021/04/05 08:57:45 restricted.go:51: Findtime: 1m0s
Fail2Ban_config: 2021/04/05 08:57:45 restricted.go:51: Ports range from 0 to 8000
Fail2Ban_config: 2021/04/05 08:57:45 restricted.go:51: FailToBan Rules : '{Xbantime:1m0s Xfindtime:1m0s Xurlregexp:[] Xmaxretry:4 Xenabled:true Xports:[0 8000]}'
Fail2Ban: 2021/04/05 08:57:45 restricted.go:52: Plugin: FailToBan is up and running

Triggering ban:

Fail2Ban: 2021/04/05 09:03:17 restricted.go:52: <external client ip> is in blacklist mode
Fail2Ban: 2021/04/05 09:03:21 restricted.go:52: <external client ip> is in blacklist mode
Fail2Ban: 2021/04/05 09:03:25 restricted.go:52: <external client ip> is in blacklist mode

Then waiting until bantime expires:

1 Fail2Ban: 2021/04/05 09:05:20 restricted.go:52: <external client ip> is now back in whitelist mode
2 Fail2Ban: 2021/04/05 09:05:33 restricted.go:52: <external client ip> is in blacklist mode
3 Fail2Ban: 2021/04/05 09:05:54 restricted.go:52: <external client ip> is in blacklist mode

When accessing Authelia after bantime expires (line 1) fail2ban correctly notifies that ip is back in whitelist mode. However, a new request (line 2) puts the client back in fail2ban mode. This does not seem right and I wonder where I have blundered in my setup?

The text was updated successfully, but these errors were encountered:

a-schaefers · 2021-04-06T02:55:29Z

Likely related to #23 (comment)

where I experience similar behavior in a different context, I'm getting banned no matter what my regexpurl is set to, and it unbans after a minute, but then with any page reload, bans again.

pittbull · 2021-04-06T06:38:25Z

Likely related to #23 (comment)

where I experience similar behavior in a different context, I'm getting banned no matter what my regexpurl is set to, and it unbans after a minute, but then with any page reload, bans again.

I agree that the ban should be lifted when bantime has passed so something is wrong, but isn't the nature of Fail2ban that a specific ip is blocked regardless of url it tries to access? My understanding is that a firewall (iptables and similar) is used and thereby blocking the ip specifically?

cledavid · 2021-04-06T21:33:40Z

Hello !
We are looking on the problem !
It can take some time so please don't mind the lapse of time.
For what I can see in the source code, the ban should be released after the minute that you specified in the configuration.

When a ip is banned, it is banned on all the domain that are registered with fail2ban so trying to access localhost:8000/one or localhost:8000/two after being banned will only reset the banning time.
We are using a table of ip with the time of their first call and a number of try. If the number of try is more than the treshold, the ip is banned. If time of the call - time of first time of calls >= findtime, the ip is debanned and should be able to access the url.

We are going to make 2 releases in the upcoming weeks :

more logs and loglevel (that will help us debug your case and upcoming ones)
regexp (allow, block, filter)

Thoses releases will help to at least debug your problem and if the problem lie in regexp it could be resolved like that !
We'll back soon with more solution so wait for us 😉
Clement

pittbull · 2021-04-07T06:07:41Z

Looking forward to your updates.

Met me know if you need any debug information.

cledavid · 2021-05-04T16:51:53Z

Hello,

We released two new versions of fail2ban : 0.6.1 & 0.6.2.
0.6.1 include an upgrade of regexp
0.6.2 add more logs
You can upgrade your version of fail2ban and check if you're still experiencing your problems, you'll have to change a bit your rules but everything is in the README.

Clement

pittbull · 2021-05-06T14:24:29Z

How would this config look in my .toml file?

tomMoulard · 2021-05-06T14:25:39Z

https://github.com/tomMoulard/fail2ban/blob/main/README.md#fail2ban

pittbull · 2021-05-06T14:52:27Z

Ok.

Updated now but situation is still the same: fail2ban basically just banning when I access my site for no specific reason. 🤷🏼‍♂️

tomMoulard · 2021-05-06T15:02:23Z

Can you produce a reproducible case ?

for no specific reason

Is not enough for me do to do something about it.

pittbull · 2021-05-06T15:24:10Z

middlewares.toml

  [http.middlewares.my-fail2ban]
    [http.middlewares.my-fail2ban.plugin]
      [http.middlewares.my-fail2ban.plugin.fail2ban]
        logLevel= "INFO"
        [http.middlewares.my-fail2ban.plugin.fail2ban.rules]
          bantime= "10s"
          enabled= true
          findtime= "60m"
          maxretry= 3
          ports= "80:443"

       [http.middlewares.my-fail2ban.plugin.fail2ban.whitelist]
         ip = ["::1", "127.0.0.1", "192.168.1.0/24"]

Activation:

INFO: Fail2Ban: 2021/05/06 17:15:33 restricted.go:51: Whitelisted: '::1/128'
INFO: Fail2Ban: 2021/05/06 17:15:33 restricted.go:51: Whitelisted: '127.0.0.1/32'
INFO: Fail2Ban: 2021/05/06 17:15:33 restricted.go:51: Whitelisted: '192.168.1.0/24'
INFO: Fail2Ban: 2021/05/06 17:15:33 restricted.go:51: Bantime: 10s
INFO: Fail2Ban: 2021/05/06 17:15:33 restricted.go:51: Findtime: 1h0m0s
INFO: Fail2Ban: 2021/05/06 17:15:33 restricted.go:51: Ports range from 80 to 443
INFO: Fail2Ban: 2021/05/06 17:15:33 restricted.go:51: FailToBan Rules : '{Xbantime:10s Xfindtime:1h0m0s XurlregexpAllow:[] XurlregexpBan:[] Xmaxretry:3 Xenabled:true Xports:[80 443]}'
INFO: Fail2Ban: 2021/05/06 17:15:33 restricted.go:52: Plugin: FailToBan is up and running

Debug log:

DEBUG: Fail2Ban: 2021/05/06 17:19:21 restricted.go:51: New request: &{GET / HTTP/1.1 1 1 map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8] Accept-Encoding:[gzip] Accept-Language:[en-us] Cdn-Loop:[cloudflare] Cf-Connecting-Ip:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] Cf-Ipcountry:[NO] Cf-Ray:[64b32fb7b9d60b6b-AMS] Cf-Request-Id:[09e3dc26d400000b6bf11bb000000001] Cf-Visitor:[{"scheme":"https"}] Connection:[Keep-Alive] Cookie:[authelia_session=vpjQsqHJtyMZtqUYWKpTQfeHuGrXrGpq; __cfduid=d4bcb3ac6988160f0689ba91f129051061620310071] User-Agent:[Mozilla/5.0 (iPhone; CPU iPhone OS 14_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1 Mobile/15E148 Safari/604.1] X-Forwarded-For:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] X-Forwarded-Host:[<mysite>] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Forwarded-Server:[a4aa097bf2e3] X-Real-Ip:[141.101.105.91]] 0xc001606600 <nil> 0 [] false <mysite> map[] map[] <nil> map[] 141.101.105.91:24982 / 0xc00140a000 <nil> <nil> 0xc000f06300}
DEBUG: Fail2Ban: 2021/05/06 17:19:21 restricted.go:52: 141.101.105.91 is no longer banned
DEBUG: Fail2Ban: 2021/05/06 17:19:21 restricted.go:51: New request: &{GET /web/index.html HTTP/1.1 1 1 map[Accept:[text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8] Accept-Encoding:[gzip] Accept-Language:[en-us] Cdn-Loop:[cloudflare] Cf-Connecting-Ip:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] Cf-Ipcountry:[NO] Cf-Ray:[64b32fb8bbad0b6b-AMS] Cf-Request-Id:[09e3dc277100000b6b140f6000000001] Cf-Visitor:[{"scheme":"https"}] Connection:[Keep-Alive] Cookie:[authelia_session=vpjQsqHJtyMZtqUYWKpTQfeHuGrXrGpq; __cfduid=d4bcb3ac6988160f0689ba91f129051061620310071] Referer:[https:// <mysite>/] User-Agent:[Mozilla/5.0 (iPhone; CPU iPhone OS 14_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1 Mobile/15E148 Safari/604.1] X-Forwarded-For:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] X-Forwarded-Host:[ <mysite>] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Forwarded-Server:[a4aa097bf2e3] X-Real-Ip:[141.101.105.91]] 0xc000ce44b0 <nil> 0 [] false  <mysite> map[] map[] <nil> map[] 141.101.105.91:36900 /web/index.html 0xc00064ea50 <nil> <nil> 0xc000ecb9b0}
DEBUG: Fail2Ban: 2021/05/06 17:19:21 restricted.go:51: welcome back 141.101.105.91 for the 2 time
DEBUG: Fail2Ban: 2021/05/06 17:19:22 restricted.go:51: New request: &{GET /web/translations/en.json HTTP/1.1 1 1 map[Accept:[application/json, text/javascript, */*; q=0.01] Accept-Encoding:[gzip] Accept-Language:[en-us] Cdn-Loop:[cloudflare] Cf-Connecting-Ip:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] Cf-Ipcountry:[NO] Cf-Ray:[64b32fba5e720b6b-AMS] Cf-Request-Id:[09e3dc287800000b6bc7971000000001] Cf-Visitor:[{"scheme":"https"}] Connection:[Keep-Alive] Cookie:[authelia_session=vpjQsqHJtyMZtqUYWKpTQfeHuGrXrGpq; __cfduid=d4bcb3ac6988160f0689ba91f129051061620310071] Referer:[https:// <mysite>/web/index.html] User-Agent:[Mozilla/5.0 (iPhone; CPU iPhone OS 14_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1 Mobile/15E148 Safari/604.1] X-Forwarded-For:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] X-Forwarded-Host:[ <mysite>] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Forwarded-Server:[a4aa097bf2e3] X-Real-Ip:[141.101.105.91] X-Requested-With:[XMLHttpRequest]] 0xc00194e618 <nil> 0 [] false  <mysite> map[] map[] <nil> map[] 141.101.105.91:45168 /web/translations/en.json 0xc0017e4000 <nil> <nil> 0xc000546e10}
DEBUG: Fail2Ban: 2021/05/06 17:19:22 restricted.go:52: 141.101.105.91 is now banned temporarily
DEBUG: Fail2Ban: 2021/05/06 17:19:22 restricted.go:51: New request: &{GET /media/providers?X-Plex-Product=Plex%20Web&X-Plex-Version=4.54.5&X-Plex-Client-Identifier=a3c11yljb5j8p0g74iucl5ww&X-Plex-Platform=Safari&X-Plex-Platform-Version=605.1&X-Plex-Sync-Version=2&X-Plex-Features=external-media%2Cindirect-media&X-Plex-Model=bundled&X-Plex-Device=iOS&X-Plex-Device-Name=Safari&X-Plex-Device-Screen-Resolution=980x1669%2C390x844&X-Plex-Token=52sp_sdeytDjRKtTf66q&X-Plex-Language=en HTTP/1.1 1 1 map[Accept:[application/json] Accept-Encoding:[gzip] Accept-Language:[en] Cdn-Loop:[cloudflare] Cf-Connecting-Ip:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] Cf-Ipcountry:[NO] Cf-Ray:[64b32fbc5a2b0b6b-AMS] Cf-Request-Id:[09e3dc29ba00000b6bd7892000000001] Cf-Visitor:[{"scheme":"https"}] Connection:[Keep-Alive] Cookie:[authelia_session=vpjQsqHJtyMZtqUYWKpTQfeHuGrXrGpq; __cfduid=d4bcb3ac6988160f0689ba91f129051061620310071] Referer:[https:// <mysite>/web/index.html] User-Agent:[Mozilla/5.0 (iPhone; CPU iPhone OS 14_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1 Mobile/15E148 Safari/604.1] X-Forwarded-For:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] X-Forwarded-Host:[ <mysite>] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Forwarded-Server:[a4aa097bf2e3] X-Real-Ip:[141.101.105.91] X-Requested-With:[XMLHttpRequest]] 0xc0016070c8 <nil> 0 [] false  <mysite> map[] map[] <nil> map[] 141.101.105.91:36900 /media/providers?X-Plex-Product=Plex%20Web&X-Plex-Version=4.54.5&X-Plex-Client-Identifier=a3c11yljb5j8p0g74iucl5ww&X-Plex-Platform=Safari&X-Plex-Platform-Version=605.1&X-Plex-Sync-Version=2&X-Plex-Features=external-media%2Cindirect-media&X-Plex-Model=bundled&X-Plex-Device=iOS&X-Plex-Device-Name=Safari&X-Plex-Device-Screen-Resolution=980x1669%2C390x844&X-Plex-Token=52sp_sdeytDjRKtTf66q&X-Plex-Language=en 0xc00064ea50 <nil> <nil> 0xc000f07650}
DEBUG: Fail2Ban: 2021/05/06 17:19:22 restricted.go:51: 141.101.105.91 is still banned since 2021-05-06T17:19:22+02:00, 4 request
DEBUG: Fail2Ban: 2021/05/06 17:19:22 restricted.go:51: New request: &{GET /media/providers?X-Plex-Product=Plex%20Web&X-Plex-Version=4.54.5&X-Plex-Client-Identifier=a3c11yljb5j8p0g74iucl5ww&X-Plex-Platform=Safari&X-Plex-Platform-Version=605.1&X-Plex-Sync-Version=2&X-Plex-Features=external-media%2Cindirect-media&X-Plex-Model=bundled&X-Plex-Device=iOS&X-Plex-Device-Name=Safari&X-Plex-Device-Screen-Resolution=980x1669%2C390x844&X-Plex-Language=en HTTP/1.1 1 1 map[Accept:[application/json] Accept-Encoding:[gzip] Accept-Language:[en] Cdn-Loop:[cloudflare] Cf-Connecting-Ip:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] Cf-Ipcountry:[NO] Cf-Ray:[64b32fbc6a4a0b6b-AMS] Cf-Request-Id:[09e3dc29c400000b6b16068000000001] Cf-Visitor:[{"scheme":"https"}] Connection:[Keep-Alive] Cookie:[authelia_session=vpjQsqHJtyMZtqUYWKpTQfeHuGrXrGpq; __cfduid=d4bcb3ac6988160f0689ba91f129051061620310071] Referer:[https:// <mysite>/web/index.html] User-Agent:[Mozilla/5.0 (iPhone; CPU iPhone OS 14_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1 Mobile/15E148 Safari/604.1] X-Forwarded-For:[2a02:2121:30c:fc3f:ad2e:cbcb:b8f3:d799] X-Forwarded-Host:[ <mysite>] X-Forwarded-Port:[443] X-Forwarded-Proto:[https] X-Forwarded-Server:[a4aa097bf2e3] X-Real-Ip:[141.101.105.91] X-Requested-With:[XMLHttpRequest]] 0xc0008b40d8 <nil> 0 [] false  <mysite> map[] map[] <nil> map[] 141.101.105.91:38766 /media/providers?X-Plex-Product=Plex%20Web&X-Plex-Version=4.54.5&X-Plex-Client-Identifier=a3c11yljb5j8p0g74iucl5ww&X-Plex-Platform=Safari&X-Plex-Platform-Version=605.1&X-Plex-Sync-Version=2&X-Plex-Features=external-media%2Cindirect-media&X-Plex-Model=bundled&X-Plex-Device=iOS&X-Plex-Device-Name=Safari&X-Plex-Device-Screen-Resolution=980x1669%2C390x844&X-Plex-Language=en 0xc00064ec60 <nil> <nil> 0xc000f062a0}
DEBUG: Fail2Ban: 2021/05/06 17:19:22 restricted.go:51: 141.101.105.91 is still banned since 2021-05-06T17:19:22+02:00, 5 request

tomMoulard · 2021-05-06T15:30:35Z

141.101.105.91 is not in ["::1", "127.0.0.1", "192.168.1.0/24"], so it works as expected

pittbull · 2021-05-06T19:19:10Z

Does that mean that the implementation of the fail2ban module is to ban all ip's not whitelisted? Because that is not what I expected based on my experience with it's namesake program.

tomMoulard · 2021-05-06T21:21:02Z

Here's the doc on how regexp works.

Feel free to open a pr that could enhance either the doc or the regexp (and the plugin) behaviour.

I will close this issue as it seems that you are asking us to build your configuration. But don't hesitate to ask your issue about it on the Traefik community forum.

cledavid added the kind/bug/confirmed Something isn't working label Apr 6, 2021

tomMoulard closed this as completed May 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fail2ban not releasing ban after expiration #34

fail2ban not releasing ban after expiration #34

pittbull commented Apr 5, 2021

a-schaefers commented Apr 6, 2021 •

edited

pittbull commented Apr 6, 2021

cledavid commented Apr 6, 2021

pittbull commented Apr 7, 2021

cledavid commented May 4, 2021

pittbull commented May 6, 2021

tomMoulard commented May 6, 2021

pittbull commented May 6, 2021 •

edited

tomMoulard commented May 6, 2021

pittbull commented May 6, 2021

tomMoulard commented May 6, 2021

pittbull commented May 6, 2021

tomMoulard commented May 6, 2021

fail2ban not releasing ban after expiration #34

fail2ban not releasing ban after expiration #34

Comments

pittbull commented Apr 5, 2021

a-schaefers commented Apr 6, 2021 • edited

pittbull commented Apr 6, 2021

cledavid commented Apr 6, 2021

pittbull commented Apr 7, 2021

cledavid commented May 4, 2021

pittbull commented May 6, 2021

tomMoulard commented May 6, 2021

pittbull commented May 6, 2021 • edited

tomMoulard commented May 6, 2021

pittbull commented May 6, 2021

tomMoulard commented May 6, 2021

pittbull commented May 6, 2021

tomMoulard commented May 6, 2021

a-schaefers commented Apr 6, 2021 •

edited

pittbull commented May 6, 2021 •

edited