irate() cuts out metric name

[RichiH]

Running the query below, the name of the metric is dropped from the result set:

So even while I have plenty of results:

They are all accumulated into one graph:

Which is not what I want here.

Playing with __name__ does not help.

As an aside, selecting on __name__ should arguably put that into my result's label set.

[RichiH]

@cron2: FYI

[brian-brazil]

This behaviour is correct, as the output of irate() is no longer the same metric. Almost all functions and operators remove the __name__ for this reason.

In this case what you need to do is explicitly select the metric names you want. Using selectors without a name should only be used for debugging and development.

[RichiH]

Even if it's only for debugging and development, an option to pass __name__ would still be useful in this context.

[RichiH]

The easy way out seems to be to add __name__ to the label set when I run {__name__=~".+",instance=~"foo.*"}.

[beorn7]

Adding __name__ is identical to adding a metric name.

Using __name__ is really considered open heart surgery with a lot of problematic side effects. We cannot introduce features to cater for its use.

Work-around for the given case: You could create a recording rule to create a new time series where the former metric name (or parts thereof) are now in a label, and then perform your original operation on that new time series.

[RichiH]

@beorn7 Agreed, that would work. It seems quite cumbersome to do that to drill down into "that's funny, let's look at all metrics related to this", though.

To phrase it differently: Is it certain that no better workaround exists/will be implemented?

[brian-brazil]

You can use label_replace, but I don't see us implementing anything in this area. It's presumed that you know your metric names when writing a query.

[beorn7]

label_replace only works on instant vectors.

I could imagine we colud make it work on range expressions, too, which would solve the problem in this particular case.

[RichiH]

@brian-brazil While this is usually a valid assumption, diving into the data to see what's there is open-ended by its very nature. And the amount of data prometheus can ingest means that people can't be expected to know everything every other team threw in there.

@beorn7 That would be appreciated. Would this realistically happen in near time or would that be longer term?

[brian-brazil]

I don't think we'd be changing label_replace in that fashion, functions only take one type of argument. You can workaround via recording rules.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.