No description, website, or topics provided.
Ruby
Clone or download
Latest commit 8e92e97 Mar 16, 2017
Permalink
Failed to load latest commit information.
gemfiles Update Fluentd dependencies Oct 28, 2016
lib/fluent/plugin support multi process workers Feb 1, 2017
test update & shrink implementation using array, enum and config_section Feb 1, 2017
.gitignore init May 11, 2012
.travis.yml update ruby versions Feb 1, 2017
Appraisals Update Fluentd dependencies Oct 28, 2016
Gemfile
LICENSE.txt v0.0.1 May 15, 2012
README.md Add requirements section Feb 3, 2017
Rakefile add test task Jul 19, 2012
fluent-plugin-notifier.gemspec

README.md

fluent-plugin-notifier

Fluentd plugin to emit notifications for messages, with numbers over/under threshold, or specified pattern strings.

Requirements

fluent-plugin-notifier fluentd ruby
>= 1.0.0 >= v0.14.0 >= 2.1
< 1.0.0 < v0.14.0 >= 1.9

Configuration

To notify apache logs with over 1000000 (microseconds) duration for CRITICAL , or status '500' by string pattern match:

<match apache.log.**>
  @type notifier
  @label @notification_events
  <def>
    pattern apache_duration
    check numeric_upward
    warn_threshold  800000
    crit_threshold 1000000
    target_keys duration
  </def>
  <def>
    pattern status_500
    check string_find
    warn_regexp 5\d\d
    crit_regexp 500
    target_key_pattern ^status.*$
    exclude_key_pattern ^status_ignore_.*$  # key name not to notify about...
  </def>
</match>

With this configuration, you will get notification messages in <label @notification_events> section, like this:

2012-05-15 19:44:29 +0900 notification: {"pattern":"apache_duration","target_tag":"apache.log.xxx","target_key":"duration","check_type":"numeric_upward","level":"crit","threshold":1000000,"value":"1057231","message_time":"2012-05-15 19:44:27 +0900"}
2012-05-15 19:44:29 +0900 notification: {"pattern":"status_500","target_tag":"apache.log.xxx","target_key":"status","check_type":"string_find","level":"crit","regexp":"/500/","value":"500","message_time":"2012-05-15 19:44:27 +0900"}

Available 'check' types: 'numeric_upward', 'numeric_downward' and 'string_find'

Default configurations:

  • tag: 'notification'
    • in top level, 'default_tag', 'default_tag_warn,' and 'default_tag_crit' available
    • in each section, 'tag', 'tag_warn' and 'tag_crit' available
  • notification suppression
    • at first, notified once in 1 minute, 5 times
    • next, notified once in 5 minutes, 5 times
    • last, notified once in 30 minutes
    • in top level, 'default_interval_1st', 'default_interval_2nd', 'default_interval_3rd', 'default_repetitions_1st' and 'default_repetitions_2nd' available
    • in each section, 'interval_1st', 'interval_2nd', 'interval_3rd', 'repetitions_1st' and 'repetitions_2nd' available

If you want to get every 5 minutes notifications (after 1 minutes notifications), specify '0' for 'repetitions_2nd'.

Message Testing

To include specified messages into check target, or to exclude specified messages from check target, directive is useful.

<match apache.log.**>
  @type notifier
  @label @notifications
  <test>
    check numeric
    target_key duration     # microseconds
    lower_threshold 5000    # 5ms
    upper_threshold 5000000 # 5s
  </test>
  <def>
    pattern status_500
    check string_find
    warn_regexp 5\d\d
    crit_regexp 500
    target_key_pattern ^status.*$
  </def>
</match>

<label @notifications>
  <match **>
    # send notifications to Slack, email or ...
  </match>
</label>

With configuration above, fluent-plugin-notifier checks messages with specified duration value (d: 5000 <= d <= 5000000), and others are ignored.

Available 'check' types are: 'numeric', 'regexp' and 'tag'.

  • numeric
    • 'lower_threshold', 'upper_threshold' and both are available
  • regexp, tag
    • 'include_pattern', 'exclude_pattern' and both are available
    • 'tag' checks tag strings after 'input_tag_remove_prefix'

Multiple directives means logical AND of each tests.

<match apache.log.**>
  @type notifier
  @label @notifications
  input_tag_remove_prefix apache.log
  <test>
    check tag
    include_pattern ^news[123]$ # for specified web server log
  </test>
  <test>
    check numeric
    target_key duration     # microseconds
    lower_threshold 5000    # 5ms
  </test>
  <test>
    check regexp
    target_key vhost
    exclude_pattern ^image.news.example.com$  # ingore image delivery server log
  </test>
  <test>
    check regexp
    target_key path
    include_pattern ^/path/to/contents/    # for specified content path only
    exclude_pattern \.(gif|jpg|png|swf)$   # but image files are ignored
  </test>
  <def>
    pattern status_500
    check string_find
    warn_regexp 5\d\d
    crit_regexp 500
    target_key_pattern ^status.*$
  </def>
</match>

Notifier plugin configured like this will check messages:

  • with tag 'apache.log.news1', 'apache.log.news2' or 'apache.log.news3'
  • with duration bigger than 5ms (upper unlimited)
  • without vhost image.news.example.com
  • with request path '/path/to/contents/*' and without file suffix gif/jpg/png/swf.

TODO

  • patches welcome!

Copyright

  • Copyright
    • Copyright (c) 2012- TAGOMORI Satoshi (tagomoris)
  • License
    • Apache License, Version 2.0