ARROW-108: [C++] Add Union implementation and IPC/JSON serialization tests #264

Closed
wants to merge 6 commits into
from

Projects

None yet

2 participants

@wesm
Member
wesm commented Jan 2, 2017

Closes #206.

Still need to add test cases for JSON read/write and dense union IPC. Integration tests can happen in a subsequent PR (but the Java library does not support dense unions yet, so sparse only -- i.e. no offsets vector)

wesm added some commits Jan 2, 2017
@wesm wesm Implement IPC/JSON serializationf or unions. Test UnionMode::SPARSE e…
…xample in IPC

Change-Id: Icc5c3f9a5080d3853ed6382f714b8a1ee2595adf
5ca9c57
@wesm wesm Move Windows stuff into a compatibility header, exclude from clang-fo…
…rmat because of include order sensitivity

Change-Id: I5376a75ca2df817570b1e6cf929bfd64e0809333
4887fd2
@xhochy

I see that you have changed ArrayPtr in a lot but not all places. The changes adds more characters but in the end, it's more readable because it actually tells one that it is a shared_ptr. But then it would be better if we would get rid of it completely.

cpp/src/arrow/array.h
+ using TypeClass = UnionType;
+
+ UnionArray(const TypePtr& type, int32_t length,
+ std::vector<std::shared_ptr<Array>>& children,
cpp/src/arrow/array.h
+ std::vector<std::shared_ptr<Array>>& children,
+ const std::shared_ptr<Buffer>& type_ids,
+ const std::shared_ptr<Buffer>& offsets = nullptr, int32_t null_count = 0,
+ std::shared_ptr<Buffer> null_bitmap = nullptr);
@xhochy
xhochy Jan 2, 2017 Member

const ref

@wesm
wesm Jan 2, 2017 Member

done

wesm added some commits Jan 2, 2017
@wesm wesm Add test case for dense union, implement RangeEquals for it
Change-Id: I69146616629ba3adcee56f63133fe9bd59922c5e
30b7188
@wesm wesm Implement basic JSON roundtrip for unions
Change-Id: If418d5862fbe84e0623c82fc74ecab5f89f86eef
3edca1e
@wesm wesm Export UnionArray
Change-Id: I02a81e35f277fb08ad54ffafec878ff9e7b025e7
cdfc61d
@wesm
Member
wesm commented Jan 2, 2017

Tests are completed

@wesm
Member
wesm commented Jan 2, 2017

Removed ArrayPtr completely

@wesm wesm Fix valgrind error
Change-Id: Ie49471df170207c6a9ce7be3601096e8f48d2204
86c4191
@wesm wesm changed the title from ARROW-108: [C++] WIP: Add Union implementation and IPC/JSON serialization tests to ARROW-108: [C++] Add Union implementation and IPC/JSON serialization tests Jan 2, 2017
@xhochy
xhochy approved these changes Jan 3, 2017 View changes

+1, LGTM

@asfgit asfgit pushed a commit that closed this pull request Jan 3, 2017
@wesm @xhochy wesm + xhochy ARROW-108: [C++] Add Union implementation and IPC/JSON serialization …
…tests

Closes #206.

Still need to add test cases for JSON read/write and dense union IPC. Integration tests can happen in a subsequent PR (but the Java library does not support dense unions yet, so sparse only -- i.e. no offsets vector)

Author: Wes McKinney <wes.mckinney@twosigma.com>

Closes #264 from wesm/ARROW-108 and squashes the following commits:

86c4191 [Wes McKinney] Fix valgrind error
cdfc61d [Wes McKinney] Export UnionArray
3edca1e [Wes McKinney] Implement basic JSON roundtrip for unions
30b7188 [Wes McKinney] Add test case for dense union, implement RangeEquals for it
4887fd2 [Wes McKinney] Move Windows stuff into a compatibility header, exclude from clang-format because of include order sensitivity
5ca9c57 [Wes McKinney] Implement IPC/JSON serializationf or unions. Test UnionMode::SPARSE example in IPC
9f7d4ae
@asfgit asfgit closed this in 9f7d4ae Jan 3, 2017
@wesm wesm deleted the wesm:ARROW-108 branch Jan 5, 2017
@pcmoritz pcmoritz referenced this pull request in ray-project/ray Jan 15, 2017
Open

Porting to upstream arrow #202

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment