[FEA] support min and max group by aggregations and reductions on lists of structs and strings

[revans2]

Is your feature request related to a problem? Please describe.
This is specifically for NVIDIA/spark-rapids#4929 for a customer query.

Describe the solution you'd like
Group by and reduction aggregations for min/max that support lists of structs and lists of strings. Ideally we could solve ordering in general like #5890 is trying to do. This would then be a follow on PR to reuse the row comparison code for min/max in this case.

If more information is needed like null ordering etc I can provide it.

Describe alternatives you've considered
Just ugly hacks that we should not do.

[devavret]

Can you explain what ordering would look like on a list of struct? Say I have

[{1, b}, {1, b}]
[{1, a}, {2, b}]

Would we compare the first child and then the second child? meaning the order of comparisons is 1, 1, a, b vs 1, 2, b, b and so idx 0 < idx 1

Or would we compare in this order 1, b, 1, b vs 1, a, 2, b meaning idx 1 < idx 0.

[revans2]

It is the second one 1, b, 1, b vs 1, a, 2, b. But more generally sorting a list in Spark looks at each element in the list starting at the first and only if they are the same does it look at the next elements to break the tie.

null == null in sorting elements, but null < non-null

If one list is longer than another list, then it shorter list is less than the longer list if and only if all of the elements in the shorter list match those in the longer list up to the length of the shorter list.

The code here is for sorting list/arrays in Spark

https://github.com/apache/spark/blob/a26c01d85035b487e2048f1c106b14b89455e2d9/sql/catalyst/src/main/scala/org/apache/spark/sql/types/ArrayType.scala#L116-L148

If you want some examples I can some up with a few.

[ttnghia]

@devavret I suppose to work on this, if nobody has been assigned.

[jrhemstad]

@ttnghia please hold off. This requires further discussion.

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[sameerz]

Sill needed

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[sameerz]

Still needed.

[ttnghia]

Depends on #11129.

[github-actions]

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.