Skip to content

Commit

Permalink
apacheGH-37437: [C++] Fix MakeArrayOfNull for list array with large s…
Browse files Browse the repository at this point in the history
…tring values type (apache#37467)

### Rationale for this change

`MakeArrayOfNull` for list type was assuming that the values child field didn't need to be considered, but those values could also require a minimum buffer size (eg for offsets) and which could be of greater size than the list offsets if those are int32 offsets.

### Are these changes tested?

Yes

* Closes: apache#37437

Authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
  • Loading branch information
jorisvandenbossche authored and dgreiss committed Feb 17, 2024
1 parent 37fb2be commit a3137b7
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 3 deletions.
5 changes: 5 additions & 0 deletions cpp/src/arrow/array/array_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,12 @@ static std::vector<std::shared_ptr<DataType>> TestArrayUtilitiesAgainstTheseType
large_utf8(),
list(utf8()),
list(int64()), // NOTE: Regression case for ARROW-9071/MakeArrayOfNull
list(large_utf8()),
list(list(int64())),
list(list(large_utf8())),
large_list(utf8()),
large_list(large_utf8()),
large_list(list(large_utf8())),
fixed_size_list(utf8(), 3),
fixed_size_list(int64(), 4),
dictionary(int32(), utf8()),
Expand Down
6 changes: 4 additions & 2 deletions cpp/src/arrow/array/util.cc
Original file line number Diff line number Diff line change
Expand Up @@ -366,9 +366,11 @@ class NullArrayFactory {
}

template <typename T>
enable_if_var_size_list<T, Status> Visit(const T&) {
enable_if_var_size_list<T, Status> Visit(const T& type) {
// values array may be empty, but there must be at least one offset of 0
return MaxOf(sizeof(typename T::offset_type) * (length_ + 1));
RETURN_NOT_OK(MaxOf(sizeof(typename T::offset_type) * (length_ + 1)));
RETURN_NOT_OK(MaxOf(GetBufferLength(type.value_type(), length_)));
return Status::OK();
}

template <typename T>
Expand Down
4 changes: 3 additions & 1 deletion cpp/src/arrow/array/validate.cc
Original file line number Diff line number Diff line change
Expand Up @@ -713,8 +713,10 @@ struct ValidateArrayImpl {
}

// An empty list array can have 0 offsets
const auto required_offsets = (data.length > 0) ? data.length + data.offset + 1 : 0;
const auto offsets_byte_size = data.buffers[1]->size();
const auto required_offsets = ((data.length > 0) || (offsets_byte_size > 0))
? data.length + data.offset + 1
: 0;
if (offsets_byte_size / static_cast<int32_t>(sizeof(offset_type)) <
required_offsets) {
return Status::Invalid("Offsets buffer size (bytes): ", offsets_byte_size,
Expand Down

0 comments on commit a3137b7

Please sign in to comment.