ARROW-658: [C++] Implement a prototype in-memory arrow::Tensor type #438
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wesm
commented
Mar 24, 2017
| @@ -43,7 +43,7 @@ class Status; | |||
| /// of bytes that where allocated for the buffer in total. | |||
| /// | |||
| /// The following invariant is always true: Size < Capacity | |||
| class ARROW_EXPORT Buffer : public std::enable_shared_from_this<Buffer> { | |||
There was a problem hiding this comment.
This is a bit of a red herring for this patch, but I ran across this cruft and decided to remove it. I can back out this change if there's concern
|
rebased |
xhochy
reviewed
Mar 26, 2017
cpp/src/arrow/tensor.h
Outdated
|
|
||
| /// Constructor with non-negative strides | ||
| Tensor(const std::shared_ptr<DataType>& type, const std::shared_ptr<Buffer>& data, | ||
| std::vector<int64_t> shape, std::vector<int64_t> strides); |
There was a problem hiding this comment.
const references for the vector attributes
cpp/src/arrow/tensor.h
Outdated
| std::vector<int64_t> shape, std::vector<int64_t> strides); | ||
|
|
||
| /// Constructor with strides and dimension names | ||
| Tensor(const std::shared_ptr<DataType>& type, const std::shared_ptr<Buffer>& data, |
Change-Id: I3b28249f15ed649abc6ddde47de451b560117b6d
Change-Id: I7df3536c1663827e720fe5de2d1110e87c52ecc3
Change-Id: I2d6adc5ff961172540177ce02e7ff8a6b14ea385
|
rebased |
cpp/src/arrow/tensor.cc
Outdated
| } | ||
|
|
||
| int64_t Tensor::size() const { | ||
| if (shape_.size() == 0) { return 0; } |
There was a problem hiding this comment.
The size of a 0-D array should probably be 1.
cpp/src/arrow/tensor.cc
Outdated
| int64_t size = 1; | ||
| for (int64_t dimsize : shape_) { | ||
| size *= dimsize; | ||
| } |
There was a problem hiding this comment.
using std::accumulate with std::multiplies should make this faster and more concise.
Change-Id: I21003a73c07ab5c71aee01dafa16a65299e85069
Change-Id: Ife060d87a629f74e83fc33690d92a3d2dfd89433
Change-Id: Ia4562e4235f8b85709b1bbb39721bba0493065c4
Change-Id: I254944a46c1231b9a78605a5bf52a1e13c09055c
|
+1, merging to take a crack at the shared memory IPC, we can continue to iterate on the Tensor data structures as needed |
wesm
pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 2, 2018
Author: Uwe L. Korn <uwelk@xhochy.com> Closes apache#438 from xhochy/PARQUET-1218 and squashes the following commits: 20ba13d [Uwe L. Korn] PARQUET-1218: More informative error message on too short pages
wesm
pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 4, 2018
Author: Uwe L. Korn <uwelk@xhochy.com> Closes apache#438 from xhochy/PARQUET-1218 and squashes the following commits: 20ba13d [Uwe L. Korn] PARQUET-1218: More informative error message on too short pages Change-Id: I3b682d1860faf49b514ba660dc1e4e7656e75122
wesm
pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 6, 2018
Author: Uwe L. Korn <uwelk@xhochy.com> Closes apache#438 from xhochy/PARQUET-1218 and squashes the following commits: 20ba13d [Uwe L. Korn] PARQUET-1218: More informative error message on too short pages Change-Id: I3b682d1860faf49b514ba660dc1e4e7656e75122
wesm
pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 7, 2018
Author: Uwe L. Korn <uwelk@xhochy.com> Closes apache#438 from xhochy/PARQUET-1218 and squashes the following commits: 20ba13d [Uwe L. Korn] PARQUET-1218: More informative error message on too short pages Change-Id: I3b682d1860faf49b514ba660dc1e4e7656e75122
wesm
pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
Author: Uwe L. Korn <uwelk@xhochy.com> Closes apache#438 from xhochy/PARQUET-1218 and squashes the following commits: 20ba13d [Uwe L. Korn] PARQUET-1218: More informative error message on too short pages Change-Id: I3b682d1860faf49b514ba660dc1e4e7656e75122
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I haven't implemented much beyond the data container and automatically computing row major strides. If we agree on the basics, then I will implement IPC read/writes of this data structure in a follow up patch.
cc @pcmoritz @robertnishihara @JohanMabille @SylvainCorlay