Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support recursive unnest #10660

Closed
duongcongtoai opened this issue May 25, 2024 · 6 comments · Fixed by #11062
Closed

Support recursive unnest #10660

duongcongtoai opened this issue May 25, 2024 · 6 comments · Fixed by #11062
Assignees
Labels
enhancement New feature or request

Comments

@duongcongtoai
Copy link
Contributor

duongcongtoai commented May 25, 2024

Is your feature request related to a problem or challenge?

These sql are not supported

##recursive unnest(list->list)
query ?
select * from unnest(unnest(
    [[1,2,3],[4,5,6]]
));
----
1
2
3
4
5
6

##recursive unnest(list->struct_field_access->list)
query ?
select * from unnest(unnest(
    [struct([1,2,3]),struct([4,5,6])]
)['c0']);
----
1
2
3
4
5
6

##recursive unnest(list->struct)
query ?
select * from unnest(unnest(
    [struct([1,2,3],[4,5,6]),struct([7,8,9],[10,11,12])]
));
----
[1,2,3] [4,5,6]
[7,8,9] [10,11,12]

If an unnest expr is a descendant of another unnest expr, this error will be thrown:

This feature is not implemented: Unnest should be rewritten to LogicalPlan::Unnest before type coercion

Describe the solution you'd like

  1. Support nested unnest expr by solving the above error.
  2. Can we support a syntax similar duckdb: within unnest expr, there is an option to specify recursion level
D select unnest([[1,2,3],[4,5,6]], max_depth:=2);
┌─────────────────────────────────────────────────────────────────────────────────────────────┐
│ unnest(main.list_value(main.list_value(1, 2, 3), main.list_value(4, 5, 6)), max_depth := 2) │
│                                            int32                                            │
├─────────────────────────────────────────────────────────────────────────────────────────────┤
│                                                                                           1 │
│                                                                                           2 │
│                                                                                           3 │
│                                                                                           4 │
│                                                                                           5 │
│                                                                                           6 │
└─────────────────────────────────────────────────────────────────────────────────────────────┘

Describe alternatives you've considered

No response

Additional context

No response

@duongcongtoai
Copy link
Contributor Author

take

@jayzhan211
Copy link
Contributor

jayzhan211 commented Jun 22, 2024

@duongcongtoai Should we just support nested unnest syntax like DuckDB? Like unnest(expr, recurisve:=true) and unnest(expr, max_depth:=2)? I noticed they do not support explicit nested unnest.

D select unnest(unnest([[1,2,3]]));
Binder Error: Nested UNNEST calls are not supported - use UNNEST(x, recursive := true) to unnest multiple levels
LINE 1: select unnest(unnest([[1,2,3]]));

@duongcongtoai
Copy link
Contributor Author

I'm thinking of supporting both, LogicalPlan for unnest can be aware of the recursion, and the recursive query can be rewritten into this LogicalPlan with a recursion_depth

@jayzhan211
Copy link
Contributor

I'm thinking of supporting both, LogicalPlan for unnest can be aware of the recursion, and the recursive query can be rewritten into this LogicalPlan with a recursion_depth

Then, we would also need to deal with somewhat complex syntax like unnest(unnest(e, recursive := true), max_depth := 2). I personally prefer having the simplest syntax for the user.

@toaiduongdh
Copy link

But for this query usecase

##recursive unnest(list->struct_field_access->list)
query ?
select * from unnest(unnest(
    [struct([1,2,3]),struct([4,5,6])]
)['c0']);

Where an operator like field access is needed between 2 unnest,it cant be supported by duckdb's syntax

@jayzhan211
Copy link
Contributor

But for this query usecase

##recursive unnest(list->struct_field_access->list)
query ?
select * from unnest(unnest(
    [struct([1,2,3]),struct([4,5,6])]
)['c0']);

Where an operator like field access is needed between 2 unnest,it cant be supported by duckdb's syntax

I see. It looks like a good reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants