Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Add metric name is a label with name "__name" to Loki output plugin #10001

Merged
merged 3 commits into from
Nov 8, 2021
Merged

fix: Add metric name is a label with name "__name" to Loki output plugin #10001

merged 3 commits into from
Nov 8, 2021

Conversation

garciarodrigor
Copy link
Contributor

Required for all PRs:

resolves #10000

Thi PR fix the loki output plugin, it add the metric name as a label with the name "__name", In order to filter data and improve queries performance in Loki.

@telegraf-tiger
Copy link
Contributor

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

@telegraf-tiger telegraf-tiger bot added the fix pr to fix corresponding bug label Oct 26, 2021
@garciarodrigor
Copy link
Contributor Author

!signed-cla

@telegraf-tiger
Copy link
Contributor

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1% increase)

📦 Looks like new artifacts were built from this PR.

Expand this list to get them here! 🐯

Artifact URLs

DEB RPM TAR GZ ZIP
amd64.deb aarch64.rpm darwin_amd64.tar.gz windows_amd64.zip
arm64.deb armel.rpm freebsd_amd64.tar.gz windows_i386.zip
armel.deb armv6hl.rpm freebsd_armv7.tar.gz
armhf.deb i386.rpm freebsd_i386.tar.gz
i386.deb ppc64le.rpm linux_amd64.tar.gz
mips.deb s390x.rpm linux_arm64.tar.gz
mipsel.deb x86_64.rpm linux_armel.tar.gz
ppc64el.deb linux_armhf.tar.gz
s390x.deb linux_i386.tar.gz
linux_mips.tar.gz
linux_mipsel.tar.gz
linux_ppc64le.tar.gz
linux_s390x.tar.gz
static_linux_amd64.tar.gz

srebhan
srebhan reviewed Oct 26, 2021
View reviewed changes
Copy link
Contributor

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @garciarodrigor, thanks for the nice PR. The code looks good. As I'm not a Loki expert I wonder what the effect of adding __name is in terms of backward compatibility. Will existing users notice the change or will all their downstream pipeline continue working?

@srebhan srebhan self-assigned this Oct 26, 2021
@srebhan srebhan added feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins and removed fix pr to fix corresponding bug labels Oct 26, 2021
@garciarodrigor
Copy link
Contributor Author

garciarodrigor commented Oct 26, 2021 •
edited

Hey @garciarodrigor, thanks for the nice PR. The code looks good. As I'm not a Loki expert I wonder what the effect of adding __name is in terms of backward compatibility. Will existing users notice the change or will all their downstream pipeline continue working?

Hi @srebhan, the only concern I have about that, is the posible colition with some existing tags named __name, I prepend the name field with __ to try to prevent that. All their downstream pipeline will continue working, this change only add a label (it is like a tag in influx world). If you have a better name or some suggest about it, it is welcome

@srebhan
Copy link
Contributor

srebhan commented Oct 27, 2021

All their downstream pipeline will continue working, this change only add a label (it is like a tag in influx world). If you have a better name or some suggest about it, it is welcome

Yeah and this might create an issue in InfluxDB because it will add another series, things that users may not want (e.g. if they have a large number of series already it will double that count). If I do understand you correctly, this is not the case with Loki. Is this correct?

@garciarodrigor
Copy link
Contributor Author

garciarodrigor commented Oct 27, 2021 •
edited

All their downstream pipeline will continue working, this change only add a label (it is like a tag in influx world). If you have a better name or some suggest about it, it is welcome

Yeah and this might create an issue in InfluxDB because it will add another series, things that users may not want (e.g. if they have a large number of series already it will double that count). If I do understand you correctly, this is not the case with Loki. Is this correct?

Loki store two different types of data: chunks and indexes, all labels (tags) will be stored in the indexes to speed up the queries, with out this missing dimesion we can't filter the data and we can incurr in some mistakes

ex:

cpu,host=xxxx usage=10 ttt
mem,host=xxxx usage=90 ttt

will be stored as

host=xxxx usage=10 ttt
host=xxxx usage=90 ttt

so, we can't run a query looking for a specific meassure or metric.

Answering your question, loki don't know the concept of "series", it have labels indexed that point to the content called "chunk"

srebhan
srebhan approved these changes Oct 27, 2021
View reviewed changes
Copy link
Contributor

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks @garciarodrigor for the PR and the helpful explanations! Btw, you do have selected a nice PR number... ;-P

@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Oct 27, 2021
@garciarodrigor
Copy link
Contributor Author

garciarodrigor commented Oct 27, 2021 •
edited

Looks good to me. Thanks @garciarodrigor for the PR and the helpful explanations! Btw, you do have selected a nice PR number... ;-P

Yeah, and a better Issue number too #10000 🥂, I haved this issue for a long time waiting for it, hahahaha

@powersj powersj merged commit e73ffe5 into influxdata:master Nov 8, 2021
powersj pushed a commit that referenced this pull request Nov 17, 2021
@garciarodrigor @powersj
fix: Add metric name is a label with name "__name" to Loki output plu…
4cbe052 
…gin (#10001)

(cherry picked from commit e73ffe5)
VladislavSenkevich pushed a commit to gwos/telegraf that referenced this pull request Nov 23, 2021
@garciarodrigor
fix: Add metric name is a label with name "__name" to Loki output plu…
41041f8 
…gin (influxdata#10001)
@powersj powersj mentioned this pull request Apr 26, 2023
Closed
powersj added a commit to powersj/telegraf that referenced this pull request Apr 27, 2023
@powersj
feat(outputs.loki): add option for metric name label
327dadf 
This allows the user to specify the label name used to store the metric
name. This label was added in influxdata#10001, but a user may not want this value
in which case they could set it to an empty string or use some other
customer value.

fixes: influxdata#13146
powersj added a commit to powersj/telegraf that referenced this pull request Apr 27, 2023
@powersj
feat(outputs.loki): add option for metric name label
4882906 
This allows the user to specify the label name used to store the metric
name. This label was added in influxdata#10001, but a user may not want this value
in which case they could set it to an empty string or use some other
customer value.

fixes: influxdata#13146
@powersj powersj mentioned this pull request Apr 27, 2023
Merged
powersj added a commit to powersj/telegraf that referenced this pull request Apr 27, 2023
@powersj
feat(outputs.loki): add option for metric name label
870be87 
This allows the user to specify the label name used to store the metric
name. This label was added in influxdata#10001, but a user may not want this value
in which case they could set it to an empty string or use some other
customer value.

fixes: influxdata#13146
powersj added a commit to powersj/telegraf that referenced this pull request May 3, 2023
@powersj
feat(outputs.loki): add option for metric name label
4c5f485 
This allows the user to specify the label name used to store the metric
name. This label was added in influxdata#10001, but a user may not want this value
in which case they could set it to an empty string or use some other
customer value.

fixes: influxdata#13146
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@powersj powersj powersj approved these changes

@srebhan srebhan srebhan approved these changes

Assignees

@srebhan srebhan

Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug]: Loki output don't include metric name as a label
3 participants
@garciarodrigor @srebhan @powersj