What is the problem the feature request solves?
When we sort an array<array<struct<...>>> or array<struct<array<...>>>, Comet falls back to Spark b/c DF doesn't support these nested types.
One of our use cases is sort_array(collect_set(named_struct())) and the result of collect_set is of type ArrayType(StructType(... ArrayType(StructType..)))
Describe the potential solution
No response
Additional context
No response
What is the problem the feature request solves?
When we sort an
array<array<struct<...>>>orarray<struct<array<...>>>, Comet falls back to Spark b/c DF doesn't support these nested types.One of our use cases is
sort_array(collect_set(named_struct()))and the result ofcollect_setis of typeArrayType(StructType(... ArrayType(StructType..)))Describe the potential solution
No response
Additional context
No response