Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for field/tag keys to strings processor #6129

Merged
merged 7 commits into from
Jul 19, 2019
Merged

Conversation

glinton
Copy link
Contributor

@glinton glinton commented Jul 16, 2019

Resolves #5928

This enables the string processor to work on tag and field keys with the config additions of tag_key and field_key.

Example

Config

[[processors.strings]]
  [[processors.strings.lowercase]]
    tag_key = "URI-Stem"
  [[processors.strings.replace]]
    tag_key = "uri-stem"
    old = "-"
    new = "_"

Input

iis_log,URI-Stem=/API/HealthCheck http_version=1.1 1519652321000000000

Output

iis_log,uri_stem=/API/HealthCheck,URI-Stem=/API/HealthCheck http_version=1.1 1519652321000000000

Try it here:

@glinton glinton added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label Jul 16, 2019
@danielnelson
Copy link
Contributor

I guess we should remove the old key after each modification? I notice that dest doesn't really make sense with keys either, which is a little unfortunate.

@glinton
Copy link
Contributor Author

glinton commented Jul 18, 2019

Should we also remove the old key when dest is specified on regular tag/field? I didn't remove the old key after each modification in order to match the behavior of dest, but agree that the old key should be removed.

@danielnelson
Copy link
Contributor

The dest operation is weird, perhaps we shouldn't include it in future processors and think about deprecating it. Maybe a copy processor would make more sense? Instead of using dest you use the copy processor, then you modify the copy. I think it will be tough to model this until we add a real language.

Looks like there are a few examples that need updated and then this is good.

@glinton
Copy link
Contributor Author

glinton commented Jul 19, 2019

For tags and fields, if dest is specified, we could add a metric.RemoveTag(key) to where we set dest = c.Dest to clean up the old keys, though now it seems it would need to be opt-in behavior.

@danielnelson
Copy link
Contributor

I'm more inclined to move in the direction of deprecating the dest option, but let's think about it and not do anything more right now.

@danielnelson danielnelson added this to the 1.12.0 milestone Jul 19, 2019
@danielnelson danielnelson merged commit bdb4598 into master Jul 19, 2019
@danielnelson danielnelson deleted the feat/5928 branch July 19, 2019 20:16
bitcharmer pushed a commit to bitcharmer/telegraf that referenced this pull request Oct 18, 2019
athoune pushed a commit to bearstech/telegraf that referenced this pull request Apr 17, 2020
idohalevi pushed a commit to idohalevi/telegraf that referenced this pull request Sep 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support strings processor operating on tag/field keys
2 participants