storage.Querier doesn't handle offsets efficiently

[jacksontj]

Bug Report

I have an implementation of the storage.Querier interface for my storage, with promql on top. In this setup if a user sends a query for a metric at a single point with an offset of 4w the querier created by promql spans 4w. This makes it difficult (to say the least) to create an efficient querier implementation. presumably similar issues exist inside prom itself (since it would potentially load significantly more data than necessary). I currently have 2 fixes for the issue, either of which I'd be glad to merge upstream-- I'm looking for some input on what fix you'd prefer.

Option 1: multiple queriers

If we don't want to change the querier interface at all we can simply make promql/engine.go create a querier for each offset. This does potentially mean more connections etc. depending on what the querier does under the hood, but does avoid the gross over-scanning of data.
Code: jacksontj@4475e2a

Option 2: add offset to selectParams

If we are okay adding a field to selectParams we can simply pass in the offset there on each actual Select. This mechanism allows for the querier to be re-used, and then makes it optional for the querier to actually look at the offset. This method would require Queriers to make code changes to pick it up, but it is backwards compatible (meaning if they don't look at the selectParam field, then they just fall back to current behavior).
Given the interface etc. I think this is the one we should do.
Code: jacksontj@cb72269

Environment

• Prometheus version:

2.2 (same issue exists in master)

[jacksontj]

We are also seeing symptoms from #3450 which may be related (since alerting rules somewhat regularly include offsets as well).

[brian-brazil]

Passing the information as a hint sounds fine to me. There's no performance impact to Prometheus with the current implementation.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.