Skip to content

/prometheus

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty labels being returned from PromQL #3841

Closed
brian-brazil opened this Issue Feb 14, 2018 · 4 comments

Comments

Assignees
No one assigned
Labels
component/service discovery kind/bug priority/P1
Projects
None yet
Milestone
  v2.2.0
3 participants
@brian-brazil @krasi-georgiev @brancz
@brian-brazil
Copy link
Member

brian-brazil commented Feb 14, 2018

prometheus/alertmanager#1240 shows empty labels somehow coming out of storage and into alerts. I've seen this previously myself also, but haven't narrowed it down. I suspect an issue on the ingestion path somewhere.

@brian-brazil brian-brazil added kind/bug component/local storage component/promql component/scraping and removed component/local storage component/promql labels Feb 14, 2018

@brian-brazil

This comment has been minimized.

Sign in to view
Copy link
Member Author

brian-brazil commented Feb 14, 2018

This is an issue in retreival by the looks of it, and honor_labels is not honoring empty strings.

@brian-brazil brian-brazil added the priority/P1 label Feb 14, 2018

@krasi-georgiev

This comment has been minimized.

Sign in to view
Copy link
Member

krasi-georgiev commented Feb 14, 2018

any easy way to replicate it?
@brian-brazil

This comment has been minimized.

Sign in to view
Copy link
Member Author

brian-brazil commented Feb 14, 2018

Pushgateway example shows it, as you end up with an instance label. Working on a fix, but this all needs unittests too.

@brian-brazil brian-brazil referenced this issue Feb 14, 2018

Closed

Fix honor_labels for empty labels, prune empty labels. #3845

@brancz brancz added this to the v2.2 milestone Feb 14, 2018

brian-brazil added a commit that referenced this issue Feb 15, 2018

@brian-brazil
Fix honor_labels for empty labels, prune empty labels. 
The semantics of honor_labels are that if a target exposes
and empty label it will override the target labels. This PR
fixes that by once again distinguishing between empty labels
and missing labels in this one use case.

Beyond that empty labels should be pruned and not added to storage,
which this also fixes.

Fixes #3841
Loading status checks…
5db6ce5

krasi-georgiev added a commit to krasi-georgiev/prometheus that referenced this issue Feb 15, 2018

@brian-brazil @krasi-georgiev
Fix honor_labels for empty labels, prune empty labels. 
The semantics of honor_labels are that if a target exposes
and empty label it will override the target labels. This PR
fixes that by once again distinguishing between empty labels
and missing labels in this one use case.

Beyond that empty labels should be pruned and not added to storage,
which this also fixes.

Fixes prometheus#3841
835df61

@krasi-georgiev krasi-georgiev referenced this issue Feb 15, 2018

Merged

Tests empty labels and honor #3849

@brian-brazil brian-brazil closed this in #3849 Feb 20, 2018

brian-brazil added a commit that referenced this issue Feb 20, 2018

@brian-brazil
Fix honor_labels for empty labels, prune empty labels. 
The semantics of honor_labels are that if a target exposes
and empty label it will override the target labels. This PR
fixes that by once again distinguishing between empty labels
and missing labels in this one use case.

Beyond that empty labels should be pruned and not added to storage,
which this also fixes.

Fixes #3841
4addee2

@brian-brazil brian-brazil added component/service discovery and removed component/scraping labels Feb 20, 2018

@lock

This comment has been minimized.

Sign in to view
Copy link

lock bot commented Mar 22, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 22, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.