New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can I parse the date not in the very beginning but in the middle of syslog? #2201

Closed
ZedYeung opened this Issue Aug 15, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@ZedYeung
Copy link

ZedYeung commented Aug 15, 2018

We will be very grateful, if your problem was described as completely as possible,
enclosing excerpts from logs (if possible within DEBUG mode, if no errors evident
within INFO mode), and configuration in particular of effected relevant settings
(e.g., with fail2ban-client -d | grep 'affected-jail-name' for a particular
jail troubleshooting).
Thank you in advance for the details, because such issues like "It does not work"
alone could not help to resolve anything!
Thanks! (remove this paragraph and other comments upon reading)

Environment:

Fill out and check ([x]) the boxes which apply. If your Fail2Ban version is outdated,
and you can't verify that the issue persists in the recent release, better seek support
from the distribution you obtained Fail2Ban from

  • Fail2Ban version (including any possible distribution suffixes):
  • OS, including release name/version:
  • [x ] Fail2Ban installed via OS/distribution mechanisms
  • [x ] You have not applied any additional foreign patches to the codebase
  • Some customizations were done to the configuration (provide details below is so)

The issue:

Aug 14 20:23:28 hostname application[345343]: 2018-08-14 20:23:28 ERROR    can not parse header when handling connection from <host>:<port>

The application output the log to /var/log/syslog, so the log would have %(__prefix_path), and then following the application log with another time stamp. Is there a way I can use the application time stamp rather than the syslog timestamp? Or just skip the __prefix_path(Aug 14 20:23:28 hostname application[345343]: )

Steps to reproduce

Expected behavior

Observed behavior

Any additional information

Configuration, dump and another helpful excerpts

Any customizations done to /etc/fail2ban/ configuration

Relevant parts of /var/log/fail2ban.log file:

preferably obtained while running fail2ban with loglevel = 4

Relevant lines from monitored log files in question:

@sebres

This comment has been minimized.

Copy link
Member

sebres commented Aug 15, 2018

Yes you can...
Just specify a datepattern explicitly, for example (?<=: )%%Y-%%m-%%d %%H:%%M:%%S
Also note that fail2ban will cut out the date-time string part from log-message before it going to match, so in this case the second date got removed, but the first date remains (so you should extend your failregex in order to match it, if ahead-anchored).

Still don't know why you want to do that (both dates look equal).

@sebres sebres closed this Aug 15, 2018

@ZedYeung

This comment has been minimized.

Copy link
Author

ZedYeung commented Aug 15, 2018

syslog doesn't contain year

I see, so that is how fail2ban works.
At first, use date pattern to capture and remove the date from log. and then use failregex to parse the log.

so for this case, I would use (.+) to match another date(is there a better regex?)
It works great! Thanks for your help!

failregex = ^.+%(__prefix_line)sERROR\s+can not parse header when handling connection from <HOST>:\d+$

ignoreregex =

datepattern = %%Y-%%m-%%d %%H:%%M:%%S
@sebres

This comment has been minimized.

Copy link
Member

sebres commented Aug 15, 2018

You are welcome.

syslog doesn't contain year

current year used in this case, if date is past, otherwise (date in the future, so impossible) the last year used (fail2ban assumes the date is past).

^.+...

this does not make sense - ^.+something... is just the same as something... (so not anchored).
Either you should use more precise regex:

-failregex = ^.+%(__prefix_line)sERROR\s+can not parse header when handling connection from <HOST>:\d+$
+failregex = ^\w+\s+\d+ \d+:\d+:\d+\s+%(__prefix_line)sERROR\s+can not parse header when handling connection from <HOST>:\d+$

datepattern = %%Y-%%m-%%d %%H:%%M:%%S

Or back to default pattern (so ignores second date via optional match of (?:\s*\d+-\d+-\d+ \d+:\d+:\d+\s+)?):

-failregex = ^.+%(__prefix_line)sERROR\s+can not parse header when handling connection from <HOST>:\d+$
+failregex = ^\s*%(__prefix_line)s(?:\s*\d+-\d+-\d+ \d+:\d+:\d+\s+)?ERROR\s+can not parse header when handling connection from <HOST>:\d+$
@ZedYeung

This comment has been minimized.

Copy link
Author

ZedYeung commented Aug 15, 2018

Awesome, it looks much better.
Thanks for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment