query:thanos-query adaptive downsample by max_source_resolution

[t00350320]

• I added CHANGELOG entry for this change.
• Change is not relevant to the end user.

Changes

here is a case: when user don't know how thanos downsample datas comes such as [raw,5m,1h]. And they may use a not suitable duration parameter [15s] like：
rate(foo[15s]& max_source_resolution=1h,
the result may be empty or only a little raw data of latest time. because the promql funcrate must need two datas accross duration time, max_source_resolution=1h, means there is only one data in 1h, the final result of rate will fail or loss data.

So , we add an adaptive method, it check the real duration time, and compared with max_source_resolution, if less than max_so.., the duration time will be changed to a suitable value.

Verification

in our systerm, we test it by set :
a: rate(foo[15s]&start1627913400&end=1628086200&step=600&max_source_resolution=1h,
b:rate(foo[15s]&start1627913400&end=1628086200&step=600&max_source_resolution=5m,

a->a1:rate(foo[2h]&start1627913400&end=1628086200&step=600&max_source_resolution=1h,
b->b1:rate(foo[10m]&start1627913400&end=1628086200&step=600&max_source_resolution=5m,
the result datas of a1 and b1 are in expectation than previous version of thanos-query.

[bwplotka]

Thanks for this!

Unfortunately, this is by design. You want to control your vector range. Normally you don't want something to change your vector range suddenly - this may surprise everyone: Suddenly you do rate over 1h not 5m. If you do increase that changes "units" kind of too.

Also, imagine you don't have downsampling but you have just long scrape interval (e.g 5m) and then you do 4m - you will also hit same issue. Also what about dynamic resolution.

What about something different: It would be nice to return error in such cases potentially (why not) if max resolution is set and unrealistic vector range is set. WDYT?

[t00350320]

yes，you are definitely right.  but i wonder there is no check of duration and max_source now. may i change the request by only return the error？

…

------------------ Original ------------------ From: Bartlomiej Plotka ***@***.***> Date: Fri,Aug 6,2021 6:29 PM To: thanos-io/thanos ***@***.***> Cc: Jupiter ***@***.***>, Author ***@***.***> Subject: Re: [thanos-io/thanos] query:thanos-query adaptive downsample by max_source_resolution (#4526) @bwplotka requested changes on this pull request. Thanks for this! Unfortunately, this is by design. You want to control your vector range. Normally you don't want something to change your vector range suddenly - this may surprise everyone: Suddenly you do rate over 1h not 5m. If you do increase that changes "units" kind of too. Also, imagine you don't have downsampling but you have just long scrape interval (e.g 5m) and then you do 4m - you will also hit same issue. Also what about dynamic resolution. What about something different: It would be nice to return error in such cases potentially (why not) if max resolution is set and unrealistic vector range is set. WDYT? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[t00350320]

hi @bwplotka , i closed this request and submit a new one
#4529.
thanks.