ARROW-442: [Python] Inspect Parquet file metadata from Python #275

Closed
wants to merge 3 commits into
from

Projects

None yet

2 participants

@wesm
Member
wesm commented Jan 9, 2017 edited

I also made the Cython parquet extension "private" so that higher level logic (e.g. upcoming handling of multiple files) can be handled in pure Python (which doesn't need to be compiled)

Requires PARQUET-828 for the test suite to pass.

wesm added some commits Jan 9, 2017
@wesm wesm Move parquet Cython wrapper to a private import, add parquet.py for h…
…igh level logic

Change-Id: I09a7a61f0ccebad4e43eec2fe2088d352f70aa2f
e59ca40
@wesm wesm Add more metadata accessor scaffolding, to be tested
Change-Id: Iad25cf265ffb15440b01e40717289a6cfba28dd5
75a11cf
@wesm
Member
wesm commented Jan 9, 2017

@xhochy I put this up before being fully done, also to let you know I'd done some refactoring in the Parquet Cython extension

@wesm wesm Add row group metadata accessor, add smoke tests
Change-Id: Id11862c108f114b93ba7941afd81203d57c11669
a4255a2
@wesm wesm changed the title from ARROW-442: [Python] WIP: Inspect Parquet file metadata from Python to ARROW-442: [Python] Inspect Parquet file metadata from Python Jan 9, 2017
@wesm
Member
wesm commented Jan 9, 2017

Removed the WIP -- @xhochy take a look when you can. thanks

@xhochy
xhochy approved these changes Jan 10, 2017 View changes

+1, LGTM

+# distutils: language = c++
+# cython: embedsignature = True
+
+from pyarrow._parquet cimport *
@xhochy
xhochy Jan 10, 2017 Member

This should not be needed?

@asfgit asfgit pushed a commit that closed this pull request Jan 10, 2017
@wesm @xhochy wesm + xhochy ARROW-442: [Python] Inspect Parquet file metadata from Python
I also made the Cython parquet extension "private" so that higher level logic (e.g. upcoming handling of multiple files) can be handled in pure Python (which doesn't need to be compiled)

Requires PARQUET-828 for the test suite to pass.

Author: Wes McKinney <wes.mckinney@twosigma.com>

Closes #275 from wesm/ARROW-442 and squashes the following commits:

a4255a2 [Wes McKinney] Add row group metadata accessor, add smoke tests
75a11cf [Wes McKinney] Add more metadata accessor scaffolding, to be tested
e59ca40 [Wes McKinney] Move parquet Cython wrapper to a private import, add parquet.py for high level logic
f44b6a3
@asfgit asfgit closed this in f44b6a3 Jan 10, 2017
@wesm wesm deleted the wesm:ARROW-442 branch Jan 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment