You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As decided in KLIP-8, continuous queries on streams should start from the beginning of the stream, not the current offsets. This causes a lot of confusion for people.
As part of this work we should disable/ignore anyone issuing SET 'auto.offset.reset'='earliest' or maybe just output a warning that this is no longer needed / heeded? We should also remove above from any docs
We should look to provide some construct to allow users to query the stream 'from now', i.e. old functionality. Some kind of WHERE offsets > current_offsets() or similar.
The text was updated successfully, but these errors were encountered:
I am +1 to this proposal. A query over a stream should always process the whole stream -- auto.offset.reset is very Kafka specific and it would be good to not use it in KSQL.
KLIP-11 includes a similar proposal. Note, that KLIP-11 proposes to handle streams and tables differently, because table records conceptually don't have an offset (the offset of records in a changelog topic are an implementation detail that should not leak into the query language).
big-andy-coates
changed the title
Pull Queries: continuous queries should start from beginning
Pull Queries: push queries should start from beginning
Nov 21, 2019
As decided in KLIP-8, continuous queries on streams should start from the beginning of the stream, not the current offsets. This causes a lot of confusion for people.
As part of this work we should disable/ignore anyone issuing
SET 'auto.offset.reset'='earliest'
or maybe just output a warning that this is no longer needed / heeded? We should also remove above from any docsWe should look to provide some construct to allow users to query the stream 'from now', i.e. old functionality. Some kind of
WHERE offsets > current_offsets()
or similar.The text was updated successfully, but these errors were encountered: