-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve GetIndexedFieldExpr adding utf8 key based access for struct v… #1204
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @Igosuki !
@houqp Maybe I should add this feature to the docs ? https://arrow.apache.org/datafusion/user-guide/sql/index.html |
@Igosuki i think it's a good idea to add it to the sql reference actually. The fact that parquet reader doesn't support nested types is not relevant to sql reference. I think it should work just fine with JSON reader at least? |
@houqp Since the JSON reader supports nested data yes it should work without issues. |
ctx.register_table("structs", table_a)?; | ||
|
||
// Original column is micros, convert to millis and check timestamp | ||
let sql = "SELECT some_struct[\"bar\"] as l0 FROM structs LIMIT 3"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is so cool!
Filed #1222 to track doc update |
Thanks again @Igosuki ! |
…alues
Which issue does this PR close?
Closes #119
Rationale for this change
Adds struct value support to select nested fields. With this, users can access arbitrary nested values in lists and structs which should cover most uses cases.
What changes are included in this PR?
GetIndexedFieldExpr gets StructArray support, allowing
select struct_col["bar"][0] from structs
Are there any user-facing changes?
Nope