Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected behavior for absent_over_time #2130

Closed
psalaberria002 opened this issue Jan 31, 2022 · 4 comments
Closed

Unexpected behavior for absent_over_time #2130

psalaberria002 opened this issue Jan 31, 2022 · 4 comments
Labels
bug Something isn't working

Comments

@psalaberria002
Copy link

Describe the bug
The absent_over_time in PromQL is expected to return 0 or 1 rows at most.
Definition taken from https://prometheus.io/docs/prometheus/latest/querying/functions/#absent_over_time:

absent_over_time(v range-vector) returns an empty vector if the range vector passed to it has any elements and a 1-element vector with the value 1 if the range vector passed to it has no elements.

VictoriaMetrics returns multiple rows in some cases, where it should not have reported any.

Expected behavior
See the following chart for example. Those are metrics from pods that are probably gone during the last few minutes, but there are still thousands of up metrics being reported. The expected result is an empty response.

Screenshot 2022-01-31 at 11 40 24

Version
v1.71.0 cluster mode

@YuriKravetc YuriKravetc added the bug Something isn't working label Feb 1, 2022
valyala added a commit that referenced this issue Feb 12, 2022
…_time()` in the same way as Prometheus does

See #2130
valyala added a commit that referenced this issue Feb 12, 2022
…_time()` in the same way as Prometheus does

See #2130
@valyala
Copy link
Collaborator

valyala commented Feb 12, 2022

@psalaberria002 , thanks for the bug report! It should be fixed in the commit 989668b . Could you build vmselect from this commit according to these docs and try executing absent_over_time() queries over this vmselect node in order to check whether the issue has been fixed?

This commit will be included in the next release of VictoriaMetrics.

@Wilderone
Copy link

Wilderone commented Feb 14, 2022

We are face this problem in process of integrating VM with kubecost.
@valyala i was build image from this commit and everything seems ok now

@valyala
Copy link
Collaborator

valyala commented Feb 14, 2022

@Wilderone , thanks for the update!

The bugfix has been integrated into VictoriaMetrics v1.73.0. Closing the issue as fixed.

@valyala valyala closed this as completed Feb 14, 2022
@psalaberria002
Copy link
Author

Thanks for the fix! Will try it in a couple of days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants