Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull Queries: Support arbitrary pull queries #3546

Closed
big-andy-coates opened this issue Oct 11, 2019 · 3 comments
Closed

Pull Queries: Support arbitrary pull queries #3546

big-andy-coates opened this issue Oct 11, 2019 · 3 comments
Labels

Comments

@big-andy-coates
Copy link
Contributor

We should support a lot more functionality for pull queries, e.g.

  • pull queries where the table is not materialized
  • pull queries with no constraint on ROWKEY
  • pull queries with JOINs or GROUP BY etc.

The distinction here is that a pull query runs from the beginning of time to current offsets. For these more complex queries we should fall back to using a KS topology against the data in Kafka i.e. using materialized state stores is an optimisation.

We can then make enhancements over time to support more and more using materialized tables.

EXPLAIN <some pull query> should make it clear if a query is a 'table scan' ie reading all data from Kafka, or using a materialized table.

@vinothchandar
Copy link
Contributor

@derekjn @apurvam @big-andy-coates Are we planning to work on this in the Q4 timeframe?

Can we prioritize them against feedback we get and schedule them for Q1 instead..

@big-andy-coates
Copy link
Contributor Author

Really, this requires #3541 first.

@big-andy-coates big-andy-coates removed their assignment Jun 19, 2020
@agavra
Copy link
Contributor

agavra commented Feb 3, 2021

Duplicate of #6726

@agavra agavra marked this as a duplicate of #6726 Feb 3, 2021
@agavra agavra closed this as completed Feb 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants