New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce support for sliding window aggregations #6873
Comments
@vcrfxia - By when can we expect this feature in ksqldb ? Actually this is very common requirements and since ksqlDB doesn't support this, we have to look for some other query engine. |
Hi @anupsdtiwari , this feature is among the items being considered for our roadmap but we don't have a specific release date to share yet. BTW, are you sure that your use case requires sliding windows rather than hopping windows? (There is some inconsistency in terminology within the stream processing field here.) Hopping windows are indeed a very common use case, but we have not seen as much demand for sliding windows. If you're willing to share your use case for sliding windows that'd be great as another data point. Thanks. |
Hi @vcrfxia - Actually sliding window seems more common at our end. My use case is as follows :- At any point of time last N hours/days/minutes/months of aggregated data is needed. Also you can refer below example :- |
Hi, I have the same requirements to @anupsdtiwari . |
Hi, There is one requirement where it is needed to trigger different resultant events as soon as a certain type of input event count crosses multiple thresholds. As soon as it crosses first threshold, we trigger event 1 and so on for upcoming thresholds... We can utilize hopping window in this use case by doing a workaround of removing duplicates, but it requires some more streams to be created and processed. By sliding window, this will be a much simpler query to emit those event. SELECT x, type, count() FROM s WINDOW SLIDING (SIZE 1 HOUR) where type="A" GROUP BY x having count()=20 EMIT CHANGES; |
Hey folks. one more use case in the sliding window will be handy: I want to calculate the crypto volume for the last 24 hours. My data looks like this: Currently, I am trying something like: But as you can see, this hopping window will create many small windows, but I want only one moving window that always gives me the result for the last 24 hours by moving the window every 1 second. |
Support for sliding window aggregations has been added to Kafka Streams (KIP-450), so we should introduce support for them in the ksqlDB syntax:
The text was updated successfully, but these errors were encountered: