Pull Query Epic

[big-andy-coates]

This issue tracks remaining work for the initial phase of pull queries

Subtasks:

• Use logical and physical query planners
• Perf test
• Support arbitrary pull queries
• Make timeouts configurable
• Pass client side property overrides through proxy
• pass any previous command id through proxy
• continuous queries should start from beginning
• document how to use standby nodes to improve availability
• Restructure code
• Add metrics
• Better formatting of output

Possible perf improvements:

• pre-compile known steps
• templated pre-compiled expected WHERE clauses

Requested enhancements:

• KEY field support
• pseudo support for structured keys
• Better error message
• Improve REST response format
• Error says table when its a stream
• Syntax change error message improvements
• Support WINDOWSTART in projection of pull queries

Known Bugs:

• ROWKEY used in expression fails
• EXPLAIN returns KS topology

Future:

• Cache arbitrary pull queries
• stream back chunked response for large data set
• include unique correlation id in processing logs
• standardize WINDOWSTART WHERE clause handling
• Suppport ROWTIME in pull queries

[apurvam]

cc @vinothchandar

[apurvam]

We may want to consider adding pagination support to lookup queries on tables. #528

[big-andy-coates]

Now tracking through https://github.com/confluentinc/ksql/projects/2