New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apache botsearch #557

Merged
merged 8 commits into from Jan 9, 2014

Conversation

Projects
None yet
3 participants
@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jan 3, 2014

Coverage Status

Coverage remained the same when pulling 7c09a61 on grooverdan:apache-botsearch into 78562fb on fail2ban:0.9.

coveralls commented Jan 3, 2014

Coverage Status

Coverage remained the same when pulling 7c09a61 on grooverdan:apache-botsearch into 78562fb on fail2ban:0.9.

@grooverdan

This comment has been minimized.

Show comment
Hide comment
@grooverdan

grooverdan Jan 7, 2014

Contributor

Any other URLs to include here?

Does the name seem right?

Contributor

grooverdan commented Jan 7, 2014

Any other URLs to include here?

Does the name seem right?

@yarikoptic

This comment has been minimized.

Show comment
Hide comment
@yarikoptic

yarikoptic Jan 7, 2014

Member

this seems to be a soup of all kinds of commits, not only apache-botsearch

Member

yarikoptic commented Jan 7, 2014

this seems to be a soup of all kinds of commits, not only apache-botsearch

@grooverdan

This comment has been minimized.

Show comment
Hide comment
@grooverdan

grooverdan Jan 7, 2014

Contributor

cleaned up like magic :-)

Contributor

grooverdan commented Jan 7, 2014

cleaned up like magic :-)

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jan 7, 2014

Coverage Status

Coverage remained the same when pulling 58ebf65 on grooverdan:apache-botsearch into bc5809e on fail2ban:0.9.

coveralls commented Jan 7, 2014

Coverage Status

Coverage remained the same when pulling 58ebf65 on grooverdan:apache-botsearch into bc5809e on fail2ban:0.9.

@yarikoptic

This comment has been minimized.

Show comment
Hide comment
@yarikoptic

yarikoptic Jan 7, 2014

Member

which ones are not caught by config/filter.d/apache-noscript.conf ?

Member

yarikoptic commented Jan 7, 2014

which ones are not caught by config/filter.d/apache-noscript.conf ?

@grooverdan

This comment has been minimized.

Show comment
Hide comment
@grooverdan

grooverdan Jan 7, 2014

Contributor

good point. raised with reporter in #544

Contributor

grooverdan commented Jan 7, 2014

good point. raised with reporter in #544

@grooverdan

This comment has been minimized.

Show comment
Hide comment
@grooverdan

grooverdan Jan 9, 2014

Contributor

I'm happy with Ivo's explaination in #544 for including this. @yarikoptic ?

Contributor

grooverdan commented Jan 9, 2014

I'm happy with Ivo's explaination in #544 for including this. @yarikoptic ?

@yarikoptic

This comment has been minimized.

Show comment
Hide comment
@yarikoptic

yarikoptic Jan 9, 2014

Member

yes -- good explanation.

I am thinking that we should start a document at some point to distil an advisory -- which filters/jails are to be used in which scenario.

For now IMHO it would be nice if this filter had a distilled sentence about that -- when this filter should be used over noscripts one. Also its failregex is to capture quite a hurd of failures but there is only a single test example. It would be nice if the collection was extended in fail2ban/tests/files/logs/apache-botsearch

Member

yarikoptic commented Jan 9, 2014

yes -- good explanation.

I am thinking that we should start a document at some point to distil an advisory -- which filters/jails are to be used in which scenario.

For now IMHO it would be nice if this filter had a distilled sentence about that -- when this filter should be used over noscripts one. Also its failregex is to capture quite a hurd of failures but there is only a single test example. It would be nice if the collection was extended in fail2ban/tests/files/logs/apache-botsearch

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jan 9, 2014

Coverage Status

Coverage remained the same when pulling 8e5366a on grooverdan:apache-botsearch into bc5809e on fail2ban:0.9.

coveralls commented Jan 9, 2014

Coverage Status

Coverage remained the same when pulling 8e5366a on grooverdan:apache-botsearch into bc5809e on fail2ban:0.9.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jan 9, 2014

Coverage Status

Coverage remained the same when pulling b0baab3 on grooverdan:apache-botsearch into bc5809e on fail2ban:0.9.

coveralls commented Jan 9, 2014

Coverage Status

Coverage remained the same when pulling b0baab3 on grooverdan:apache-botsearch into bc5809e on fail2ban:0.9.

grooverdan added a commit that referenced this pull request Jan 9, 2014

Merge pull request #557 from grooverdan/apache-botsearch
ENH: Apache botsearch + BF: tag substition

@grooverdan grooverdan merged commit 8333abe into fail2ban:0.9 Jan 9, 2014

1 check passed

default The Travis CI build passed
Details

@grooverdan grooverdan deleted the grooverdan:apache-botsearch branch Jan 9, 2014

return False
else:
if tags.has_key(found_tag):
value = value[0:m.start()] + tags[found_tag] + value[m.end():]
value = value.replace('<%s>' % found_tag , tags[found_tag])

This comment has been minimized.

@yarikoptic

yarikoptic Jan 10, 2014

Member

although much more straightforward, it might be less efficient since in previous implementation there is no more search to do, no string interpolations -- just plain concatenation of strings using already determined positions... in big realm of things -- probably would be least to metter

@yarikoptic

yarikoptic Jan 10, 2014

Member

although much more straightforward, it might be less efficient since in previous implementation there is no more search to do, no string interpolations -- just plain concatenation of strings using already determined positions... in big realm of things -- probably would be least to metter

This comment has been minimized.

@yarikoptic

yarikoptic Jan 10, 2014

Member

ah commit says that it was buggy in dealing with multiple tags on the line... need to run with your nice test now since can't see where the bug was

@yarikoptic

yarikoptic Jan 10, 2014

Member

ah commit says that it was buggy in dealing with multiple tags on the line... need to run with your nice test now since can't see where the bug was

This comment has been minimized.

@grooverdan

grooverdan Jan 10, 2014

Contributor

It would fail on "failregex = <param> x=<ip> <param>" trying to substitute the second param as it though its already done that and suspected a recursion.

@grooverdan

grooverdan Jan 10, 2014

Contributor

It would fail on "failregex = <param> x=<ip> <param>" trying to substitute the second param as it though its already done that and suspected a recursion.

This comment has been minimized.

@yarikoptic

yarikoptic Jan 10, 2014

Member

ah... so it is a 'find-a-recursion' which fails then if substitution is not done for all instances at once... cool ;-)

@yarikoptic

yarikoptic Jan 10, 2014

Member

ah... so it is a 'find-a-recursion' which fails then if substitution is not done for all instances at once... cool ;-)

This comment has been minimized.

@grooverdan

grooverdan Jan 10, 2014

Contributor

thinking a bit more there may be some other failure causes along this line where the second param is contained within another def. Time for more test cases.

@grooverdan

grooverdan Jan 10, 2014

Contributor

thinking a bit more there may be some other failure causes along this line where the second param is contained within another def. Time for more test cases.

This comment has been minimized.

@yarikoptic

yarikoptic Jan 10, 2014

Member

On Thu, 09 Jan 2014, Daniel Black wrote:

  •                                      value = value[0:m.start()] + tags[found_tag] + value[m.end():]
    
  •                                      value = value.replace('<%s>' % found_tag , tags[found_tag])
    

thinking a bit more there may be some other failure causes along this line
where the second param is contained within another def. Time for more test
cases.

yeah... I had a feeling that current recursion detection look a bit
"incomplete", but could not see exactly how, thus kept quiet ;)

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

@yarikoptic

yarikoptic Jan 10, 2014

Member

On Thu, 09 Jan 2014, Daniel Black wrote:

  •                                      value = value[0:m.start()] + tags[found_tag] + value[m.end():]
    
  •                                      value = value.replace('<%s>' % found_tag , tags[found_tag])
    

thinking a bit more there may be some other failure causes along this line
where the second param is contained within another def. Time for more test
cases.

yeah... I had a feeling that current recursion detection look a bit
"incomplete", but could not see exactly how, thus kept quiet ;)

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

This comment has been minimized.

@yarikoptic

yarikoptic Jan 10, 2014

Member

ideally we should properly inspect for recursion, i.e. loops in a graph
among tags (first build adjacency matrix, i.e. which tag uses which
tag), and then use one of those
http://en.wikipedia.org/wiki/Strongly_connected_components
to detect if there is any loop (strongly connected component)

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

@yarikoptic

yarikoptic Jan 10, 2014

Member

ideally we should properly inspect for recursion, i.e. loops in a graph
among tags (first build adjacency matrix, i.e. which tag uses which
tag), and then use one of those
http://en.wikipedia.org/wiki/Strongly_connected_components
to detect if there is any loop (strongly connected component)

Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik

# Webroot represents the webroot on which all other files are based
webroot = /var/www/
# Block is the actual non-found directories to block
block = (<webmail>|<phpmyadmin>|<wordpress>)[^,]*

This comment has been minimized.

@yarikoptic

yarikoptic Feb 3, 2015

Member

@grooverdan would you remember why you have decided to plug all those into explicit variables here instead of just defining them for string interpolations, how we did in common.conf for other generic common patterns?

@yarikoptic

yarikoptic Feb 3, 2015

Member

@grooverdan would you remember why you have decided to plug all those into explicit variables here instead of just defining them for string interpolations, how we did in common.conf for other generic common patterns?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment