feat(expr): support array_distinct

[snipekill]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Feature #7765

Checklist For Contributors

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

Types of user-facing changes

• SQL commands, functions, and operators

Release note

Support array_distinct.

[lmatz]

Thank you for your contribution!
Could you add some e2e tests and/or unit tests for array_distinct https://github.com/risingwavelabs/risingwave/tree/main/e2e_test/batch/functions

[snipekill]

Thank you for your contribution! Could you add some e2e tests and/or unit tests for array_distinct https://github.com/risingwavelabs/risingwave/tree/main/e2e_test/batch/functions

Yeah sure

[snipekill]

@lmatz To get distinct values, I am using Hashset, as it is unordered, writing the output for e2e tests can be tricky, as output order of elements in the array can change. Do, I sort the input and use dedup instead, it would not be that performant, and I can sort the output after.

[xiangjinwu]

@lmatz To get distinct values, I am using Hashset, as it is unordered, writing the output for e2e tests can be tricky, as output order of elements in the array can change. Do, I sort the input and use dedup instead, it would not be that performant, and I can sort the output after.

There is a way to get deterministic output with HashSet rather than sorting: scan the array and skip those already in HashSet, add new values to output and HashSet.

https://docs.rs/itertools/0.10.5/itertools/trait.Itertools.html#method.unique

[snipekill]

@lmatz To get distinct values, I am using Hashset, as it is unordered, writing the output for e2e tests can be tricky, as output order of elements in the array can change. Do, I sort the input and use dedup instead, it would not be that performant, and I can sort the output after.

There is a way to get deterministic output with HashSet rather than sorting: scan the array and skip those already in HashSet, add new values to output and HashSet.

https://docs.rs/itertools/0.10.5/itertools/trait.Itertools.html#method.unique

Makes sense will change the logic

[snipekill]

@fuyufjh @lmatz have made all the required changes, added unit and e2e tests as well, can you check the code pertaining to type checking of inputs to match function signature.

[xiangjinwu]

Rest lgtm. Thank you!

[fuyufjh]

Good job! Thanks a lot for your contribution! @snipekill