Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LATEST considers a bucket whose start time doesn't fall within [fromTimestamp, toTimestamp] #1486

Closed
OwsOwies opened this issue Jul 6, 2023 · 2 comments

Comments

@OwsOwies
Copy link

OwsOwies commented Jul 6, 2023

Hi, having an issue with LATEST. Having an hourly compaction series with the following data:

$ redis-cli TS.RANGE key - + LATEST
1) 1) (integer) 1671087600000
   2) 18500000000
2) 1) (integer) 1671091200000
   2) 7700000000
3) 1) (integer) 1671624000000
   2) 700000000
4) 1) (integer) 1671627600000
   2) 5900000000
5) 1) (integer) 1672650000000
   2) 60700000000
6) 1) (integer) 1672653600000
   2) 106500000000
7) 1) (integer) 1673967600000 // This is not here without LATEST
   2) 2400000000

Making the following query includes the latest compacted value whose start time doesn't fall within [fromTimestamp, toTimestamp].

$ redis-cli TS.RANGE key 1685570400000 1688162399999 LATEST
1) 1) (integer) 1673967600000
   2) 2400000000

From: Jun 01 2023
To: Jun 30 2023
Returned: Jan 17 2023

What's wrong?

On a side note I'd also like a clarification on what ALIGN does:

$ redis-cli TS.RANGE key fromTimestamp toTimestamp LATEST AGGREGATION SUM {toTimestamp - fromTimestamp} ALIGN fromTimestamp

Would this query correctly aggregate ONLY values of timestamps between fromTimestamp to toTimestamp into a single bucket considering ALL points which fall in between from and to? The documentation on ALIGN is a bit ambiguous to me.
As I understand, without ALIGN keyword, there could be multiple buckets, not really falling in between from and to, so I would like to clarify that ALIGN serves my use case.

Thanks!

@LiorKogan
Copy link
Member

LiorKogan commented Jul 6, 2023

This seems like a bug. We will try to reproduce it.

Regarding ALIGN:
Your assumptions are correct.
By default, aggregation buckets of one hour will contain data for [00:00 .. 01:00), [01:00 .. 02:00) etc.
But suppose you want the buckets to be [00:30 .. 01:30), [01:30 .. 02:30) etc. - you can align the buckets to 00:30 after the Epoch (or on any other date). That way, you force one aggregation bucket to start at the specified time, and all other buckets are aligned accordingly.

AGGREGATION SUM {toTimestamp - fromTimestamp} ALIGN fromTimestamp should indeed create a single bucket for the period between fromTimestamp and toTimestamp. Maybe you should use AGGREGATION SUM {toTimestamp - fromTimestamp + 1} ALIGN fromTimestamp, please try this if you receive two buckets.

Note that when aggregating over a compacted time series, the timestamps represent the start timestamp of the raw buckets.

OfirMos pushed a commit that referenced this issue Jul 6, 2023
…ater then the lastest sample it shouldn't be includede in the result
OfirMos pushed a commit that referenced this issue Jul 6, 2023
…ater then the lastest sample it shouldn't be includede in the result
OfirMos pushed a commit that referenced this issue Jul 6, 2023
…ater then the lastest sample it shouldn't be includede in the result
OfirMos pushed a commit that referenced this issue Jul 6, 2023
…ater then the lastest sample it shouldn't be includede in the result

(cherry picked from commit 16f7103)
OfirMos pushed a commit that referenced this issue Jul 6, 2023
…ater then the lastest sample it shouldn't be includede in the result

(cherry picked from commit 16f7103)
OfirMos pushed a commit that referenced this issue Jul 6, 2023
…ater then the lastest sample it shouldn't be includede in the result

(cherry picked from commit 16f7103)
OfirMos pushed a commit that referenced this issue Jul 9, 2023
…ater then the lastest sample it shouldn't be includede in the result

(cherry picked from commit 16f7103)
@LiorKogan
Copy link
Member

Fixed in RedisTimeSeries 1.8.11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants