You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
From the mailing list discussion, we need to define the structure of data headers to describe a "page" of Arrow data. In Apache Parquet these are referred to as the data page headers and are separate from the schema metadata.
As part of this JIRA:
Choose a technology (it need not be the only one, but it would be good to have a reference tool) for representing the data page header in "serialized" form (as can be stored in shared memory and interpreted by any system with access to that memory). Google's Flatbuffers libraries has been discussed for this use case.
From the mailing list discussion, we need to define the structure of data headers to describe a "page" of Arrow data. In Apache Parquet these are referred to as the data page headers and are separate from the schema metadata.
As part of this JIRA:
Choose a technology (it need not be the only one, but it would be good to have a reference tool) for representing the data page header in "serialized" form (as can be stored in shared memory and interpreted by any system with access to that memory). Google's Flatbuffers libraries has been discussed for this use case.
Draft IDL specification for the data headers
Reporter: Wes McKinney / @wesm
Assignee: Wes McKinney / @wesm
Note: This issue was originally created as ARROW-57. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: