You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This doesn't seem to be an issue with the master branch as of opening this issue, so it might not be severe enough to try force into the 3.0.0 release.
I tested null and non-null Spark files, and was able to read them correctly. This becomes an issue with nested lists, which I'm working on.
We currently do not propagate child nullability correctly when reading parquet files from Spark 3.0.1 (parquet-mr 1.10.1).
For example, the below taken from https://github.com/apache/parquet-format/blob/master/LogicalTypes.md is currently interpreted incorrectly:
The Arrow type should be:
but we currently end up with
This doesn't seem to be an issue with the master branch as of opening this issue, so it might not be severe enough to try force into the 3.0.0 release.
I tested null and non-null Spark files, and was able to read them correctly. This becomes an issue with nested lists, which I'm working on.
Reporter: Neville Dipale / @nevi-me
Assignee: Neville Dipale / @nevi-me
PRs and other links:
Note: This issue was originally created as ARROW-11271. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: