Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for reading JSON Array to Arrow #4905

Closed
mnpw opened this issue Oct 9, 2023 · 2 comments · Fixed by #4911
Closed

Support for reading JSON Array to Arrow #4905

mnpw opened this issue Oct 9, 2023 · 2 comments · Fixed by #4911
Labels
arrow Changes to the arrow crate question Further information is requested

Comments

@mnpw
Copy link

mnpw commented Oct 9, 2023

Which part is this question about

Reading from JSON to Arrow (arrow_json::reader module)

Describe your question

arrow_json::reader mentions

This JSON reader allows JSON line-delimited files to be read into the Arrow memory model...

I'm curious as to why does the reader not support Array JSON type ([{...}, {...}, {...}, ...])? I found that arrow2 is able to do so but not arrow-rs.

Additional context
N/A
@mnpw mnpw added the question Further information is requested label Oct 9, 2023
@tustvold
Copy link
Contributor

tustvold commented Oct 9, 2023

It's a consequence of the formulation of the reader in terms of RecordBatch and by extension Schema, which correspond to JSON structs. The reader supports lists, just currently not at the root of the schema.

This could probably be relaxed without too much difficulty, is this important to your use-case or just an observation?

tustvold added a commit to tustvold/arrow-rs that referenced this issue Oct 9, 2023
@tustvold
Support Arbitrary JSON values in JSON Reader (apache#4905)
e355b8a
@tustvold tustvold mentioned this issue Oct 9, 2023
Merged
tustvold added a commit that referenced this issue Oct 12, 2023
@tustvold
Support Arbitrary JSON values in JSON Reader (#4905) (#4911)
90bc5ec 
* Support Arbitrary JSON values in JSON Reader (#4905)

* Review feedback

* Clippy

* Docs
@tustvold tustvold added the arrow Changes to the arrow crate label Oct 18, 2023
@tustvold
Copy link
Contributor

label_issue.py automatically added labels {'arrow'} from #4911

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arrow Changes to the arrow crate question Further information is requested
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support Arbitrary JSON values in JSON Reader (#4905) tustvold/arrow-rs
2 participants
@tustvold @mnpw