Skip to content

Commit

Permalink
Require "remove_tag_prefix" to be specified in configs. (#78)
Browse files Browse the repository at this point in the history
Without this option specified, the output of the plugin will be fed back to
itself, causing infinite recursion and errors that can be hard to understand.
  • Loading branch information
faximan committed Jun 18, 2020
1 parent a32c00c commit b1f57bc
Show file tree
Hide file tree
Showing 3 changed files with 40 additions and 16 deletions.
16 changes: 10 additions & 6 deletions README.rdoc
Expand Up @@ -42,19 +42,23 @@ will also install and configure the gem.

The plugin supports the following parameters:

=== Required

[remove_tag_prefix] The prefix to remove from the input tag when outputting
a record. A prefix has to be a complete tag part.
Example: If remove_tag_prefix is set to 'foo', the input
tag foo.bar.baz is transformed to bar.baz and the input tag
'foofoo.bar' is not modified.

=== Optional

[message] Name of the field in the JSON record that contains the
single-line log messages that shall be scanned for exceptions.
If this is set to '', the plugin will try 'message' and 'log',
in that order.
This parameter is only applicable to structured (JSON) log streams.
Default: ''.

[remove_tag_prefix] The prefix to remove from the input tag when outputting
a record. A prefix has to be a complete tag part.
Example: If remove_tag_prefix is set to 'foo', the input
tag foo.bar.baz is transformed to bar.baz and the input tag
'foofoo.bar' is not modified. Default: empty string.

[languages] A list of language for which exception stack traces shall be
detected. The values in the list can be separated by commas or
written as JSON list.
Expand Down
4 changes: 2 additions & 2 deletions lib/fluent/plugin/out_detect_exceptions.rb
Expand Up @@ -22,10 +22,10 @@ module Fluent
# an exception stack trace, they forwarded as a single, combined JSON
# object. Otherwise, the input log data is forwarded as is.
class DetectExceptionsOutput < Output
desc 'The prefix to be removed from the input tag when outputting a record.'
config_param :remove_tag_prefix, :string
desc 'The field which contains the raw message text in the input JSON data.'
config_param :message, :string, default: ''
desc 'The prefix to be removed from the input tag when outputting a record.'
config_param :remove_tag_prefix, :string, default: ''
desc 'The interval of flushing the buffer for multiline format.'
config_param :multiline_flush_interval, :time, default: nil
desc 'Programming languages for which to detect exceptions. Default: all.'
Expand Down
36 changes: 28 additions & 8 deletions test/plugin/test_out_detect_exceptions.rb
Expand Up @@ -28,6 +28,8 @@ def setup

DEFAULT_TAG = 'prefix.test.tag'.freeze

DEFAULT_TAG_STRIPPED = 'test.tag'.freeze

ARBITRARY_TEXT = 'This line is not an exception.'.freeze

JAVA_EXC = <<END.freeze
Expand Down Expand Up @@ -125,7 +127,9 @@ def test_ignore_nested_exceptions
}

test_cases.each do |language, exception|
cfg = "languages #{language}"
cfg = %(
#{CONFIG}
languages #{language})
d = create_driver(cfg)
t = Time.now.to_i

Expand Down Expand Up @@ -156,12 +160,12 @@ def test_ignore_nested_exceptions

# Validate that each line received is emitted separately as expected.
router_mock.should_receive(:emit)
.once.with(DEFAULT_TAG, Integer,
.once.with(DEFAULT_TAG_STRIPPED, Integer,
'message' => json_line_with_exception,
'count' => 0)

router_mock.should_receive(:emit)
.once.with(DEFAULT_TAG, Integer,
.once.with(DEFAULT_TAG_STRIPPED, Integer,
'message' => json_line_without_exception,
'count' => 1)

Expand All @@ -174,7 +178,9 @@ def test_ignore_nested_exceptions
end

def test_single_language_config
cfg = 'languages java'
cfg = %(
#{CONFIG}
languages java)
d = create_driver(cfg)
t = Time.now.to_i
d.run do
Expand All @@ -185,7 +191,9 @@ def test_single_language_config
end

def test_multi_language_config
cfg = 'languages python, java'
cfg = %(
#{CONFIG}
languages python, java)
d = create_driver(cfg)
t = Time.now.to_i
d.run do
Expand All @@ -196,7 +204,9 @@ def test_multi_language_config
end

def test_split_exception_after_timeout
cfg = 'multiline_flush_interval 1'
cfg = %(
#{CONFIG}
multiline_flush_interval 1)
d = create_driver(cfg)
t1 = 0
t2 = 0
Expand Down Expand Up @@ -227,6 +237,12 @@ def test_do_not_split_exception_after_pause
assert_equal(make_logs(t1, JAVA_EXC + " at x\n at y\n"), d.events)
end

def test_remove_tag_prefix_is_required
cfg = ''
e = assert_raises(Fluent::ConfigError) { create_driver(cfg) }
assert_match(/remove_tag_prefix/, e.message)
end

def get_out_tags(remove_tag_prefix, original_tag)
cfg = "remove_tag_prefix #{remove_tag_prefix}"
d = create_driver(cfg, original_tag)
Expand All @@ -244,7 +260,9 @@ def test_remove_tag_prefix
end

def test_flush_after_max_lines
cfg = 'max_lines 2'
cfg = %(
#{CONFIG}
max_lines 2)
d = create_driver(cfg)
t = Time.now.to_i
d.run do
Expand All @@ -263,7 +281,9 @@ def test_flush_after_max_lines
end

def test_separate_streams
cfg = 'stream stream'
cfg = %(
#{CONFIG}
stream stream)
d = create_driver(cfg)
t = Time.now.to_i
d.run do
Expand Down

0 comments on commit b1f57bc

Please sign in to comment.