feat(agg,over_window): support first_value, last_value and refactor LogicalAgg

[stdrc]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

• Support last_value aggregate function and hence corresponding window function
• Expose first_value and last_value to user
• Refactor LogicalAgg to make match arms clean and also allow more 2-phase agg kind

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• [ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• All checks passed in ./risedev check (or alias, ./risedev c)
• [ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

• [ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the [details](https://github.com/risingwavelabs/risingwave/blob/main/CONTRIBUTING.md))

Documentation

• My PR contains user-facing changes.

Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

• SQL commands, functions, and operators

Release note

• Support first_value and last_value aggregate function and window function

[kwannoel]

Sqlsmith part LGTM

[kwannoel]

Looks there might be some bugs introduced though, see fuzzing-1.log in build artifacts - https://buildkite.com/risingwavelabs/pull-request/builds/26777#01892404-766d-4c48-a47f-49224f2ee6e0:

2022-10-11T05:44:26.555094Z  INFO node{id=7 name="frontend-1"}:task{id=561}:run_stmt{stmt=CREATE MATERIALIZED VIEW m14 AS SELECT approx_count_distinct('5D1ZlMzATx') AS col_0 FROM m8 AS t_0 WHERE true GROUP BY t_0.col_0, t_0.col_2 HAVING true session_id=0}: risingwave_expr::sig::agg: 211 aggregations loaded.
thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
  left: `2`,
 right: `3`', src/meta/src/stream/stream_graph/fragment.rs:279:9
stack backtrace:

CREATE MATERIALIZED VIEW m14 AS
SELECT
    approx_count_distinct ('5D1ZlMzATx') AS col_0
FROM
    m8 AS t_0
WHERE
    TRUE
GROUP BY
    t_0.col_0,
    t_0.col_2
HAVING
    TRUE

[kwannoel]

Shouldn't approx count distinct fall under this category? 🤔

[kwannoel]

And StringAgg too?

[kwannoel]

Seems to be the source of errors.

[stdrc]

I guess the src/meta/src/stream/stream_graph/fragment.rs:279:9 is because visit_stream_node_tables_inner didn't correctly visit state table of DistinctApproxCount, will dive in later. While OTOH 2-phase sum0(hyperloglog) may indeed not equal to hyperloglog, not quite sure. I'll add it to this category.

For StringAgg, I was thinking that when user doesn't provide ORDER BY, the call can actually be 2-phased by just concatenating strings in arbitrary nondeterministic order.

[kwannoel]

For StringAgg, I was thinking that when user doesn't provide ORDER BY, the call can actually be 2-phased by just concatenating strings in arbitrary nondeterministic order.

Agree on correctness, although I cant think of a usecase.
I suppose there's some other bug around it then. Maybe it is executing in non-append-only. We should add another check for that.

An e2e test will be great to test two-phase string_agg as well, I don't think it is currently tested. Same goes for any other two-phase agg we have just started supporting in this PR.

[stdrc]

Seems 2-phase StringAgg has other problem, that is the final phase can't get delim argument from input. Maybe better to disable it again for now😇.

[wangrunji0408]

Maybe we can infer or allow developers to specify these properties in the #[aggregate] macro later. 🤔

[codecov]

Codecov Report

Merging #10740 (d6e9ffb) into main (2732fef) will increase coverage by 0.00%.
The diff coverage is 65.24%.

@@           Coverage Diff           @@
##             main   #10740   +/-   ##
=======================================
  Coverage   70.14%   70.15%           
=======================================
  Files        1296     1296           
  Lines      221698   221748   +50     
=======================================
+ Hits       155516   155569   +53     
+ Misses      66182    66179    -3     

Flag	Coverage Δ
rust	70.15% <65.24%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/expr/src/agg/def.rs	67.96% <0.00%> (-0.67%)	⬇️
...c/frontend/src/optimizer/rule/distinct_agg_rule.rs	98.76% <ø> (ø)
src/common/src/util/stream_graph_visitor.rs	51.53% <16.66%> (-1.97%)	⬇️
src/expr/src/agg/general.rs	85.26% <20.00%> (-0.86%)	⬇️
...rc/frontend/src/optimizer/plan_node/generic/agg.rs	84.27% <63.01%> (-0.95%)	⬇️
src/stream/src/executor/aggregation/minput.rs	95.80% <87.50%> (-0.09%)	⬇️
src/common/src/util/sort_util.rs	96.34% <100.00%> (+1.55%)	⬆️
src/frontend/src/expr/agg_call.rs	74.16% <100.00%> (+0.21%)	⬆️
...rc/frontend/src/optimizer/plan_node/logical_agg.rs	96.65% <100.00%> (-0.02%)	⬇️

... and 12 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more