ARROW-5111: [Go] implement reading list arrays from Arrow file #4157
Conversation
|
needs #4155 |
Codecov Report
@@ Coverage Diff @@
## master #4157 +/- ##
===========================================
- Coverage 87.89% 60.65% -27.24%
===========================================
Files 758 67 -691
Lines 92163 4883 -87280
Branches 1251 0 -1251
===========================================
- Hits 81006 2962 -78044
+ Misses 11040 1812 -9228
+ Partials 117 109 -8Continue to review full report at Codecov.
|
go/arrow/ipc/file_reader.go
Outdated
| @@ -409,6 +425,29 @@ func (ctx *arrayLoaderContext) loadPrimitive(dt arrow.DataType) array.Interface | |||
| return array.MakeFromData(data) | |||
| } | |||
|
|
|||
| func (ctx *arrayLoaderContext) loadBinary(dt arrow.DataType) array.Interface { | |||
| field, buffers := ctx.loadCommon(3) | |||
There was a problem hiding this comment.
Hi @sbinet, I'm working on the equivalent for Rust. Hope you don't mind me asking a few questions.
- Does the binary array always consist of 3 buffers, or would it be 2 if the null count is 0? Here's the relevant code in the Rust impl: https://github.com/apache/arrow/pull/4167/files#diff-cd3519a5e748548b5323b0d05d8e9c8aR53 (cc @pitrou @kszucs )
- Do you mind sharing a script for generating Arrow data in Python (I took the one you used when you asked a question in the mailing list)? I haven't used pyarrow in a while, so I was struggling with making lists and structs.
Thanks
There was a problem hiding this comment.
Thanks @pitrou, is it safe for me to presume that the buffer numbers for primitives, list and struct will behave in the same way, where the null buffer may be null or empty?
There was a problem hiding this comment.
Yes, it's part of the Arrow format specification.
There was a problem hiding this comment.
@nevi-me here is what I am using:
https://gist.github.com/sbinet/8ec09eb95bc89c6d71cc192dfcce677b
No description provided.