Skip to content

Update query endpoint to use new pipeline executor #3015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Mar 31, 2023

Conversation

jachris
Copy link
Contributor

@jachris jachris commented Mar 14, 2023

This PR updates the /query REST endpoint to use the new pipeline API. This also means that queries are not allowed to start with an expression. Use where instead.

It also adds operator_base::predicate_pushdown such that queries such as where x == 1 | where y == 2 | head 5 are optimized into head 5, with an additional x == 1 && y == 2 constraint that is passed to the index.

@jachris jachris force-pushed the topic/complete-logical-operators branch from caa9b2f to ad6fb58 Compare March 16, 2023 12:11
Base automatically changed from topic/complete-logical-operators to main March 16, 2023 17:33
@jachris jachris force-pushed the topic/update-query-pipeline branch 4 times, most recently from 794d5bb to 24987da Compare March 20, 2023 09:48
@jachris jachris marked this pull request as ready for review March 20, 2023 09:54
This commit also adds `logical_operator::predicate_pushdown`.
@jachris jachris force-pushed the topic/update-query-pipeline branch from 24987da to 1ae39c5 Compare March 20, 2023 11:03
jachris and others added 3 commits March 20, 2023 12:38
Due to incomplete support by the current CI compilers.
The `put` operator is a replacement for both the `extend` and the
`replace` operators. It takes as arguments a comma-separated list of
assignments. If the assigned-to extractor resolves for the input data
replaces all occurences of the field (previously `replace`), and if the
extractor does not resolve adds one new field at the end (previously
`extend`).

`put` is the first operator not to be available in the
old pipeline operator framing, and as such is not currently reachable
by the user.
Copy link
Member

@dominiklohmann dominiklohmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great piece of work. I tested it out extensively locally as part of the put PR (which is based on this).

Approving as-is, but I've got one minor thing that I think we could change or write a follow-up issue for.

dominiklohmann and others added 14 commits March 28, 2023 16:41
Co-authored-by: Jannis Christopher Köhl <mail@koehl.dev>
The `put` operator is a replacement for both the `extend` and the
`replace` operators. It takes as arguments a comma-separated list of
assignments. If the assigned-to extractor resolves for the input data
replaces all occurences of the field (previously `replace`), and if the
extractor does not resolve adds one new field at the end (previously
`extend`).

`put` is the first operator not to be available in the old pipeline
operator framing, and as such is not currently reachable by the user.
@jachris jachris merged commit dae0bd8 into main Mar 31, 2023
@jachris jachris deleted the topic/update-query-pipeline branch March 31, 2023 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants