[feature] Add PromQL support for querying metrics at an absolute time

[ncabatoffims]

When investigating anomalies it would be helpful to be able to issue queries relative to a specific point in time. For example, when I was trying to determine what was causing the odd histogram_quantile graphs which led me to file #1887, I was trying to see the raw data for the anomalous data point in the console. The process was iterative as I refined my query and tried different ways to make sense of the data. Each time I tried a new query, I would often find that the offset had to be updated, e.g.

test1 offset 53m

think, read docs

test2 offset 53m

nope, more than 1m passed

test2 offset 54m
test2 offset 55m

think, read docs

test3 offset 55m
test3 offset 56m

It's annoying. I'd like to be able to simply say

test1 @ 1470941522

or maybe even

test1 @ 20160811T185803Z

I understand the use case for wanting to get metrics from an absolute point in time isn't a very important one, but it would be a nice convenience.

[juliusv]

@ncabatoffims Would this be solved by this UI feature? #1246

[ncabatoffims]

Not really, because sometimes I want to look at the raw data in text form rather than the graph. I can just use Grafana if I want to have a static window of time in which to experiment with graphs.

[juliusv]

I think #1246 is about that - allowing timestamp also for instant queries in the console.

[brian-brazil]

I'm not sure this belongs in PromQL itself, wanting to compare a range against an absolute vs relative time slice that we currently support doesn't seem like something that you'd want to do and may cause confusion.

[ncabatoffims]

I don't have a strong desire for it be in PromQL itself, I'd be fine with it being a UI feature. However #1246 doesn't say or (from my reading) even imply that it would also apply to console queries. If you want to close this issue as a dup could we make that explicit in #1246?

[juliusv]

I think #1246 only makes sense in the context of console / instant queries since graphs allow you to specify the exact time range already. So yeah, it's about the console view. Would be an easy feature to add, as the backend support for that is already there.

[ncabatoffims]

Ah, you're right, shows how little I use the Prometheus Graph feature. Closing as dup.

[sharpensteel]

i was able to get raw data on specific time using API request:

http://localhost:9090/api/v1/query?query=nodejs_eventloop_lag_seconds[20s]&time=2017-10-30T08:54:00Z

there is no visible options to specify time in console:

[flaviostutz]

I wanted to create an alert that compares the current metrics to another in a specific point in time. In this case, this specific point in time was a time when I determined everything was normal. How would I do this in a query? Any ideas?

[juliusv]

@flaviostutz I don't think there's a way to do that in a single PromQL alerting rule as long as it's about an absolute time and not a relative one.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.