New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support split of tag #8

Merged
merged 6 commits into from Oct 17, 2013

Conversation

Projects
None yet
2 participants
@futoase
Contributor

futoase commented Oct 11, 2013

Support split of tag.

An proposal.
This difference is split tag, use of period.

${tag} #== hoge.fuga.moge

${tags[0]} #== hoge
${tags[1]} #== fuga
${tags[2]} #== moge

fluentd.conf(sample)

<source>
  type forward
  port 24244
</source>

<match app.game.*>
  type rewrite_tag_filter
  rewriterule1 server_type ^([a-z]+)$ ${tag[1]}.$1.${tag[2]}
</match>

<match game.production.api>
  type copy
  <store>
    type tdlog
    api_key hogehogehogehogehogehoge
  </store>

  <store>
    type file
    path /var/log/fluentd/production_api.log
  </store>
</match>

<match game.staging.api>
  type copy
  <store>
    type mongo
  </store>
  <store>
    type file
    path /var/log/fluentd/staging_api.log
  </store>
</match>

<match game.development.api>
  type file
  path /var/log/fluentd/development_api.log
</match>

command

> echo '{"user_id": "10000", "name": "hogehoge", "server_type": "development", "description": "super user"}' | bundle ex fluent-cat app.game.peach
> echo '{"user_id": "10000", "name": "hogehoge", "server_type": "production", "description": "super user"}' | bundle ex fluent-cat app.game.mario
> echo '{"user_id": "10000", "name": "hogehoge", "server_type": "staging", "description": "super user"}' | bundle ex fluent-cat app.game.luigi

result

  • production_api.log
2013-10-11T15:25:28+09:00       game.production.mario   {"user_id":"10000","name":"hogehoge","server_type":"production","description":"super user"}
  • staging_api.log
2013-10-11T15:25:48+09:00       game.staging.luigi      {"user_id":"10000","name":"hogehoge","server_type":"staging","description":"super user"}
  • development_api.log
2013-10-11T15:25:08+09:00       game.development.peach  {"user_id":"10000","name":"hogehoge","server_type":"development","description":"super user"}
@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 11, 2013

Owner

tag.split('.').each_with_index でもいいような気がした

Owner

futoase commented on lib/fluent/plugin/out_rewrite_tag_filter.rb in 9651a69 Oct 11, 2013

tag.split('.').each_with_index でもいいような気がした

@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 11, 2013

Owner

スコープ外のtagと名前が被る、t のがいいのだろうか

Owner

futoase commented on lib/fluent/plugin/out_rewrite_tag_filter.rb in 9651a69 Oct 11, 2013

スコープ外のtagと名前が被る、t のがいいのだろうか

@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 11, 2013

Owner

Change method from Hash#merge! to Hash#store.

Owner

futoase commented on lib/fluent/plugin/out_rewrite_tag_filter.rb in affad29 Oct 11, 2013

Change method from Hash#merge! to Hash#store.

@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 11, 2013

Owner

Use tag.split('.').

Owner

futoase commented on lib/fluent/plugin/out_rewrite_tag_filter.rb in affad29 Oct 11, 2013

Use tag.split('.').

@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 11, 2013

Owner

remove this config_param.

Owner

futoase commented on test/plugin/test_out_rewrite_tag_filter.rb in affad29 Oct 11, 2013

remove this config_param.

@y-ken

This comment has been minimized.

Show comment
Hide comment
@y-ken

y-ken Oct 11, 2013

Member

to: @repeatedly @tagomoris
Would you please review for this pull-req to keep compatibility for another plugins?

Member

y-ken commented Oct 11, 2013

to: @repeatedly @tagomoris
Would you please review for this pull-req to keep compatibility for another plugins?

@y-ken

This comment has been minimized.

Show comment
Hide comment
@y-ken

y-ken Oct 11, 2013

Member

to: @futoase
would you please update the pull-req description?
It has been got still old information included like split_of_tag and ${tag[0]}.

Member

y-ken commented Oct 11, 2013

to: @futoase
would you please update the pull-req description?
It has been got still old information included like split_of_tag and ${tag[0]}.

@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 11, 2013

Contributor

It has been got still old information included like split_of_tag and ${tag[0]}.

oh...

Contributor

futoase commented Oct 11, 2013

It has been got still old information included like split_of_tag and ${tag[0]}.

oh...

@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 11, 2013

Contributor

updated!

Contributor

futoase commented Oct 11, 2013

updated!

@y-ken

This comment has been minimized.

Show comment
Hide comment
@y-ken

y-ken Oct 12, 2013

Member

On adding array range specify which does not supported yet like ${tags[0..2]}, it will be ignored.
Could you please add checking format at configure to raise ConfigError like below?

if cond
  raise Fluent::ConfigError, "${tags} placeholder does not support range specify at #{key} #{conf[key]}"
end
Member

y-ken commented Oct 12, 2013

On adding array range specify which does not supported yet like ${tags[0..2]}, it will be ignored.
Could you please add checking format at configure to raise ConfigError like below?

if cond
  raise Fluent::ConfigError, "${tags} placeholder does not support range specify at #{key} #{conf[key]}"
end
@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 13, 2013

Contributor

Could you please add checking format at configure to raise ConfigError like below?

OK. I respond to an this proposal.

Contributor

futoase commented Oct 13, 2013

Could you please add checking format at configure to raise ConfigError like below?

OK. I respond to an this proposal.

@y-ken

This comment has been minimized.

Show comment
Hide comment
@y-ken

y-ken Oct 14, 2013

Member

rewriterule1 server_type ^([a-z]+)$ ${tag[1]}.$1.${tag[2]}

Oops! it still got outdated placeholder at the pull-req description...
And also, It is welcome to update the README.md at the section of #tag-placeholder.

It should be affected on this page when you have pushing new commit to your branch of support-split-of-tag

Member

y-ken commented Oct 14, 2013

rewriterule1 server_type ^([a-z]+)$ ${tag[1]}.$1.${tag[2]}

Oops! it still got outdated placeholder at the pull-req description...
And also, It is welcome to update the README.md at the section of #tag-placeholder.

It should be affected on this page when you have pushing new commit to your branch of support-split-of-tag

@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 14, 2013

Contributor

And also, It is welcome to update the README.md at the section of #tag-placeholder.

OK, I will update the README.md.

Contributor

futoase commented Oct 14, 2013

And also, It is welcome to update the README.md at the section of #tag-placeholder.

OK, I will update the README.md.

@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 15, 2013

Contributor

#8 (comment)

This comment proposal I finished.

Contributor

futoase commented Oct 15, 2013

#8 (comment)

This comment proposal I finished.

@y-ken

This comment has been minimized.

Show comment
Hide comment
@y-ken

y-ken Oct 15, 2013

it might be s/tag/tags/g aren't you?

y-ken commented on README.md in ec0b66d Oct 15, 2013

it might be s/tag/tags/g aren't you?

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 15, 2013

Owner

Sorry, I repair it.

Owner

futoase replied Oct 15, 2013

Sorry, I repair it.

@y-ken

This comment has been minimized.

Show comment
Hide comment
@y-ken

y-ken Oct 17, 2013

Member

Great! Thank you for your pull-req.
I will gem release after behavior test has finished on my real environment.

Member

y-ken commented Oct 17, 2013

Great! Thank you for your pull-req.
I will gem release after behavior test has finished on my real environment.

y-ken added a commit that referenced this pull request Oct 17, 2013

Merge pull request #8 from futoase/support-split-of-tag
Enhanced placeholder like ${tags[n]} which adds part of tag splitting by dot without any settings. Thanks @futoase !

**example**
`apache.access` tag would be resolved like below.
${tags[0]} => apache
${tags[1]} => access

**note**
This function has added as default behavior.

@y-ken y-ken merged commit f5b9301 into fluent:master Oct 17, 2013

1 check passed

default The Travis CI build passed
Details
@futoase

This comment has been minimized.

Show comment
Hide comment
@futoase

futoase Oct 18, 2013

Contributor

Thank you!!! 😸

Contributor

futoase commented Oct 18, 2013

Thank you!!! 😸

@futoase futoase deleted the futoase:support-split-of-tag branch Oct 18, 2013

@y-ken

This comment has been minimized.

Show comment
Hide comment
@y-ken

y-ken Mar 10, 2014

Member

It has renamed placeholder from 'tags' to 'tag_parts' in f5b9301...7a57c60 commit.

Member

y-ken commented Mar 10, 2014

It has renamed placeholder from 'tags' to 'tag_parts' in f5b9301...7a57c60 commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment