Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rust] [Parquet] Support v2 LogicalType #27654

Closed
Tracked by #27258
asfimport opened this issue Feb 27, 2021 · 1 comment
Closed
Tracked by #27258

[Rust] [Parquet] Support v2 LogicalType #27654

asfimport opened this issue Feb 27, 2021 · 1 comment

Comments

@asfimport
Copy link

We currently do not read nor write the version 2 logical types. This is mainly because we do not have a mapping for it from parquet-format-rs.

To implement this, we can:

  • convert "parquet::basic::LogicalType" to "parquet::basic::ConvertedType"

  • implement "parquet::basic::LogicalType" which mirrors "parquet_format::LogicalType"

  • create a mapping between ConvertedType and LogicalType

  • write LogicalType to "parquet_format::SchemaElement" if v2 of the writer is used

    This would be a good starting point for implementing 2.6 types (UUID, NANOS precision time & timestamp).
    Follow-up work would be:

  • parsing v2 of the schema [ARROW-11365]

  • Using v2 in the Arrow writer (mostly schema conversion)

  • Supporting nanosecond precision time & timestamp

Reporter: Neville Dipale / @nevi-me
Assignee: Neville Dipale / @nevi-me

PRs and other links:

Note: This issue was originally created as ARROW-11803. Please see the migration documentation for further details.

@asfimport
Copy link
Author

Neville Dipale / @nevi-me:
Issue resolved by pull request 9592
#9592

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants