Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out_file: add warn message for symlink_path setting #4502

Conversation

Shingo-Nakayama
Copy link
Contributor

@Shingo-Nakayama Shingo-Nakayama commented May 23, 2024

Which issue(s) this PR fixes:

What this PR does / why we need it:
According to the issue #4490 , if a tag or keys is set in the chunk key and those placeholders do not exist in the symlink_path, multiple chunks to compete for the same symlink and some logs can't be taken from it.
Since this is unintended behavior, I think it is necessary to inform the user that this is a bad setting by the warn message.

If a tag or keys is set in the chunk key and those placeholders do not exist in the symlink_path, the following warning log is displayed.

Ex:

<source>
  @type sample
  sample {"message":"hoge","key1":"A","key2":"B"}
  tag test.hoge
</source>

<source>
  @type sample
  sample {"message":"fuga","key1":"A","key2":"B"}
  tag test.fuga
</source>

<match test.**>
  @type file
  path /tmp/test/${tag}/${key1}/${key2}
 symlink_path /tmp/test/link
  append true

  <buffer time,tag,key1,key2>
    @type file
    path /tmp/test/buffer/tag
    timekey 60 
    timekey_wait 3
  </buffer>
</match>
[warn]: #0 Parameter 'symlink_path: /tmp/test/link' doesn't have tag placeholder. This means multiple chunks are competing for a single symlink_path, so some logs may not be taken from the symlink.
[warn]: #0 Parameter 'symlink_path: /tmp/test/link' doesn't have enough placeholders for keys key1,key2. This means multiple chunks are competing for a single symlink_path, so some logs may not be taken from the symlink.

Docs Changes:
None (may be necessary)

Release Note:
Same with the title

Signed-off-by: Shingo Nakayama <nkymsng@akane.waseda.jp>
Copy link
Contributor

@daipom daipom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this fix!
Could you please fix the following points?

lib/fluent/plugin/out_file.rb Outdated Show resolved Hide resolved
lib/fluent/plugin/out_file.rb Outdated Show resolved Hide resolved
test/plugin/test_out_file.rb Outdated Show resolved Hide resolved
test/plugin/test_out_file.rb Show resolved Hide resolved
test/plugin/test_out_file.rb Outdated Show resolved Hide resolved
Signed-off-by: Shingo Nakayama <nkymsng@akane.waseda.jp>
Co-authored-by: Daijiro Fukuda <fukuda@clear-code.com>
@Shingo-Nakayama Shingo-Nakayama force-pushed the out_file-add-warn-message-for-symlink_path-setting branch from 5b85ac8 to 1a64c6e Compare May 24, 2024 06:44
Copy link
Contributor

@daipom daipom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@daipom daipom added this to the v1.16.6 milestone May 24, 2024
@daipom daipom merged commit 74b2e3d into fluent:master May 29, 2024
15 of 16 checks passed
@daipom daipom changed the title out_file:add warn message for symlink_path-setting out_file: add warn message for symlink_path setting May 29, 2024
daipom added a commit that referenced this pull request May 29, 2024
Backported from 74b2e3d.

Signed-off-by: Shingo Nakayama <nkymsng@akane.waseda.jp>
Co-authored-by: Daijiro Fukuda <fukuda@clear-code.com>
daipom added a commit that referenced this pull request May 29, 2024
Backported from 74b2e3d.

Signed-off-by: Shingo Nakayama <nkymsng@akane.waseda.jp>
Signed-off-by: Daijiro Fukuda <fukuda@clear-code.com>
Co-authored-by: Daijiro Fukuda <fukuda@clear-code.com>
daipom added a commit that referenced this pull request May 29, 2024
Backported from 74b2e3d. (#4502) 

Signed-off-by: Shingo Nakayama <nkymsng@akane.waseda.jp>
Signed-off-by: Daijiro Fukuda <fukuda@clear-code.com>
Co-authored-by: Shingo Nakayama <152375941+Shingo-Nakayama@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

out_file:Inconsistent path settings and symlink_path settings.
2 participants