Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load logpath only after findtime is configured #2173

Merged
merged 2 commits into from Jul 6, 2018
Merged

Load logpath only after findtime is configured #2173

merged 2 commits into from Jul 6, 2018

Conversation

@mattsta
Copy link
Contributor

@mattsta mattsta commented Jul 5, 2018

Commit Message

When new log paths are configured, their start offset is immediately determined
by a filter searching for (now - findTime).
But, since findTime is configured after the log is loaded and
searched, logs are only searched back by the default 10 minute findTime,
regardless of user configuration of jail settings.

So, findTime must be configured before logpath or else the default findtime
is used, which ignores any findtime time defined by the user.

This fixes new reads on startup for actual log files. The systemd filter
always performed as expected due to being setup after the jail's
findtime config submission.

Story Time!

It took me probably 10 hours to track down this problem. Spent a long time thinking regexes were just broken for half my jails because lookback-on-load worked reliably on some jails, but not others. Turns out the reliable jails were all systemd and loading from files was kinda broken. Then took a while to figure out how configs actually get loaded.

Additional Fixes Maybe

The newer config options logtimezone and logencoding are also loaded after the logfile is initially processed. If those are used to read the log, they should also be configured before logpath gets sent to the transmitter, otherwise default class values will always be used instead of configured values.

and, one minor change that may have helped cut hours off these debuggles: surfacing the initial lookback results in INFO mode instead of deep debug mode (here and here) would be nice for users to always know (since it's the start point for actually searching for bad actions on startup).

@coveralls
Copy link

@coveralls coveralls commented Jul 5, 2018

Coverage Status

Coverage remained the same at 97.404% when pulling 1eb93e2 on mattsta:fix/findtime-backsearch-on-file-load into 857d695 on fail2ban:0.10.

@codecov-io
Copy link

@codecov-io codecov-io commented Jul 5, 2018

Codecov Report

Merging #2173 into 0.10 will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@           Coverage Diff           @@
##             0.10    #2173   +/-   ##
=======================================
  Coverage   95.57%   95.57%           
=======================================
  Files          76       76           
  Lines       13190    13190           
  Branches     2081     2082    +1     
=======================================
  Hits        12607    12607           
  Misses        315      315           
  Partials      268      268
Impacted Files Coverage Δ
fail2ban/client/jailreader.py 93.91% <ø> (ø) ⬆️
fail2ban/server/filter.py 95.75% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 857d695...1eb93e2. Read the comment docs.

mattsta and others added 2 commits Jul 5, 2018
When new log paths are configured, their start offset is immediately determined
by a filter searching for (now - findTime).
But, since findTime is configured *after* the log is loaded and
searched, logs are only searched back by the default 10 minute findTime,
regardless of user configuration of jail settings.

So, findTime must be configured before logpath or else the default findtime
is used, which ignores any findtime time defined by the user.

This fixes new reads on startup for actual log files. The systemd filter
always performed as expected due to being setup after the jail's
findtime config submission.
…position of `findtime` option in config);

jailreader.py: additionally relocate the option `logpath` after all log-related data (backend, date-pattern, etc) that may be needed by the first usage (gh-2173).
Thanks to Matt Stancliff (mattsta)
@sebres sebres changed the base branch from 0.11 to 0.10 Jul 6, 2018
@sebres
Copy link
Contributor

@sebres sebres commented Jul 6, 2018

I've rebased this to 0.10 (because affected also) and made a small amendment to cover all possible constellations.
Will merge hereafter if all test-cases passed.
Thank you for your contribution!

@sebres sebres merged commit 1eb93e2 into fail2ban:0.10 Jul 6, 2018
3 checks passed
3 checks passed
codecov/patch 100% of diff hit (target 95.57%)
Details
codecov/project 95.57% (+0%) compared to 857d695
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
sebres added a commit that referenced this pull request Jul 6, 2018
…-load (rebased)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants