-
|
I installed fail2ban on ubuntu 22.04 LOMP (OpenLiteSpeed) The installation seems to go well and fail2ban seems to work fine. May 10 20:29:25 vicetemple-lomp systemd[1]: Started Fail2Ban Service. But I see bots passing through my access.log that should be blocked by fail2ban, but no ban on the fail2ban logs... This is my jail.local and this is the filter that should block bots Did I miss something? |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 2 replies
-
|
You have to provide access-log excerpt, so one can check what exactly doesn't match. Also note https://github.com/fail2ban/fail2ban/wiki/How-fail2ban-works And https://github.com/fail2ban/fail2ban/wiki/Best-practice#reduce-parasitic-log-traffic |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, I've thought about it a bit these days, maybe this: These are the apache2 access.logs These are openlitespeed access.log This is the regular expression of the filter This filter has worked perfectly for years on Apache2, so I think the cause is different access.log formats. Can the filter understand website1, website2, etc? (regular expressions are not my strong point) I also have some strange warnings in the fail2ban logs... P.S. maybe its easy change the log format in openlitespeed? |
Beta Was this translation helpful? Give feedback.
-
|
I don't see whether the messages should be matched by stock apache-badbots (due to agents that are normally not in - failregex = ^<HOST> -.*"(GET|POST).*HTTP.*".*(?:%(badbots)s|%(badbotscustom)s).*"$
+ failregex = ^(?:\[[^\]]*\] )?<HOST> -.*"(GET|POST).*HTTP.*".*(?:%(badbots)s|%(badbotscustom)s).*"$However this filter is (and always was) a bit strange - (a bit vulnerable due to mant catch-alls, checks only for get and post methods, etc). failregex = ^(?:\[[^\]]*\] )?<ADDR> [^"]*"[^"]*" \d+ \S+ "[^"]*" "[^"]*(?:%(badbots)s|%(badbotscustom)s)[^"]*"$But better would be not to use filters like this at all:
|
Beta Was this translation helpful? Give feedback.
-
|
Yes, it is a strange old filter that I have also been using for many years, and in fact it has been manipulated in many ways, even probably incorrect ones. but it works... somehow... what would you use to throw out the badbots? |
Beta Was this translation helpful? Give feedback.
-
It is complex topic, so too long to describe... Let alone it depends on the definition what one calling as "bad" or which goals exactly you want to achieve. Here are few thoughts about similar subjects:
There is also no common generic scenario (again, because it depends on many factors)... Fo example, if the goal is something like limiting of requests or connections for bots, better option could be good concepted robots.txt and a rate-limiting for bots (e. g. nginx has 2 modules for that - limit_req and limit_conn) and/or filters like nginx-limit-req to ban too aggressive bots by exceeding limits (or generating a lot of traffic on unexpected locations). If the goal is to prevent some bots to scan your web-site for some content, it is hardly possible without a whole complex of measures, starting with user behavior modeling, inclusive real user detection (like cookies, behavior algorithms, captcha, etc) and ending with rate limits and other complicated rules on the side of web-server, backend and/or frontend. And it is not and will never be a generalized solution. Anyway the idea to ban a bot just by means of its name in access-log is ugly and doomed to fail. |
Beta Was this translation helpful? Give feedback.
I don't see whether the messages should be matched by stock apache-badbots (due to agents that are normally not in
badbots, neither inbadbotscustom)...But OK... to match second format also, you have to rewrite the RE:
However this filter is (and always was) a bit strange - (a bit vulnerable due to mant catch-alls, checks only for get and post methods, etc).
I'd use something like that instead:
B…