[DYOD] Support window function queries #2607

Upstream said that we should create new `Chunk`s that re-use the previous segments (they are stored as shared pointers).

…nctionEvaluator::partition_and_sort()`

…dowFunction::Rank`

…pute_window_function`

Assigning does not work, because `operator[]` returns `AllTypeVariant`, not a reference.

It appears that someone is renaming our column after the operator has executed.

…it uses multiple partition-by columns

This is needed to prevent creating a chunk that has both `ReferenceSegment`s and `ValueSegments`s, as our output column is always represented by `ValueSegment`s.

We now reuse all segments from the input table. If the input table is a reference table, we create a new table with our output value segments and then add reference segments to this new table in our output table.

… Assert for correct rank frame

…escription`

This adds `_templated_on_execute` so that the input column type and window function are compile time constants the whole time. This way, we don't get template instantiation errors down the line when we resolve types again.

Squashed commit of the following: commit 60864ca Author: Finn Schoellkopf <schoellkopf.finn@gmail.com> Date: Tue Jun 27 19:31:30 2023 +0200 add helper functions for calculating frame_bounds commit a0ffb9f Author: Finn Schoellkopf <schoellkopf.finn@gmail.com> Date: Tue Jun 27 19:08:24 2023 +0200 change null handling of sum commit 6cf5efe Author: Finn Schoellkopf <schoellkopf.finn@gmail.com> Date: Tue Jun 27 18:58:27 2023 +0200 fix unbounded preceding/following Frambounds commit d33f3d8 Author: Finn Schoellkopf <schoellkopf.finn@gmail.com> Date: Tue Jun 27 18:32:46 2023 +0200 add support for Row FrameBounds for Sum

…:optional` cannot be used as template argument

Squashed commit of the following: commit f4e0ef2404b86f3b78455297abe85a51e1d87b29 Author: Niklas Mohrin <dev@niklasmohrin.de> Date: Thu Jun 29 16:36:04 2023 +0200 Finish up the work for passing `RelevantRowInformation` around instead of materialized input commit acba832 Author: Magnus Menger <magnus@magnusmenger.de> Date: Wed Jun 28 15:09:21 2023 +0200 start getting rid of partitioned data and using RelevantRowInformationEverywhere (not working yet) Co-authored-by: Niklas Mohrin <dev@niklasmohrin.de>

…ion`

…_pass, segment_tree}`

… fix incorrect computation

…r variable

The window functions now have to specify the type of the segment tree node and what to do with the query result. For simple window functions, this can be achieved by using a `using` for the node type and importing the `IdentityTransformer` into the traits scope. More involved functions, such as AVG can however specify arbitrary functions.

This also changes the comparator to use `std::weak_ordering` instead of `std::partiai_ordering`, because all our values are actually comparable. I didn't go for `std::strong_ordering`, because I am unsure whether the required invariant holds for us and the distinction between "equivalent" and "equal" don't really matter for the usecase anyways.

…ed; add constructors for onepass with initial row

…ash_function`s that also want to get the partition's hash value

For TPC-DS query 51, which is the longest-running supported one, this makes the operator only take half of the time just by adding the parallelism in the initial sorting step. The merging does not seem to have a significant impact, but I added it for completeness.

…oolean to make the pqp more readable

…ecause we want to separate window bound calculation for Rows and Range mode

This also needed a little refactor, because we need the type of the order-by column, but partial function specialization is not allowed, so we use a type instead.

From postgres documentation: The default framing option is RANGE UNBOUNDED PRECEDING, which is the same as RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW. With ORDER BY, this sets the frame to be all rows from the partition start up through the current row's last ORDER BY peer. Without ORDER BY, this means all rows of the partition are included in the window frame, since all rows become peers of the current row. So we just return the entire partition if we know that no order-by column was specified.

This needed a refactor away from `TreeNode` constructors to dedicated factory functions, because the constructors where clashing for the Count implementation. I think the `window_function_combinator.hpp` file has grown such that a rewrite with the current requirements in mind would be reasonable.

…ator_traits.hpp`

… strategies

… the `window_function_evaluator` namespace

…e separate steps anyways)

We want to use features that gcc 9 doesn't support. Commands to reproduce: ```sh sed -i 's/gcc-9/gcc-10/g' Jenkinsfile Dockerfile install_dependencies.sh sed -i 's/g++-9/g++-10/g' Jenkinsfile Dockerfile install_dependencies.sh sed -i 's/gcc9/gcc10/g' Jenkinsfile Dockerfile install_dependencies.sh sed -i 's/g++9/g++10/g' Jenkinsfile Dockerfile install_dependencies.sh ```

… map

…t them :^)

…unequal orderings

…ng of private variables.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DYOD] Support window function queries #2607

[DYOD] Support window function queries #2607

Commits on Aug 17, 2023

Commits on Aug 18, 2023

Commits on Aug 19, 2023

Commits on Aug 20, 2023

Commits on Aug 21, 2023

Commits on Aug 22, 2023

Commits on Aug 23, 2023

Commits on Aug 24, 2023

[DYOD] Support window function queries #2607

Are you sure you want to change the base?

[DYOD] Support window function queries #2607

Commits on Aug 17, 2023

Commits on Aug 18, 2023

Commits on Aug 19, 2023

Commits on Aug 20, 2023

Commits on Aug 21, 2023

Commits on Aug 22, 2023

Commits on Aug 23, 2023

Commits on Aug 24, 2023