Change in selector behaviour for !=

[tcolgate]

What did you do?
query:
metrics{label != "thing"}
where label is not set.

What did you expect to see?
In prometheus 1.6.3 this will return all the items where label != "thing", including items where label is not set at all.

What did you see instead? Under which circumstances?
In prometheus 2.0 items without label set are not returned.

[gouthamve]

Yep, this is confirmed and doesn't look like a trivial change. Will work on this whenever I get time, but cannot commit to anything.

So, essentially this here is not selecting stuff which doesn't have a label set: https://github.com/prometheus/tsdb/blob/master/querier.go#L257-L306

One of the proposals is that we could materialise a view which has the postings for all series where l != v and then use that. But that also means that the matcher interface we have doesn't cut it anymore.

We could also not bleed this into tsdb and rather make Prometheus make 2 queries, one with "absent" matcher, one with "!=" and then merge the results of the two.

/cc @krasi-georgiev

[brian-brazil]

We could also not bleed this into tsdb and rather make Prometheus make 2 queries, one with "absent" matcher, one with "!=" and then merge the results of the two.

I don't think this should leak, and there could be multiple such selectors.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.