[C++] arrow::io::CompressedInputStream might allocate buffer to many times

[mapleFU]

Describe the enhancement requested

in ::arrow::io::CompressedInputStream::Impl, the decompressed_buffer_ is initialized as 1MB, and would:

1. Call Resize to release the memory if possible ( https://github.com/apache/arrow/blob/main/cpp/src/arrow/io/compressed.cc#L303 )
2. Droped the memory when buffer exhausted ( https://github.com/apache/arrow/blob/main/cpp/src/arrow/io/compressed.cc#L324 )

Can we avoid re-allocating the buffer frequently, trying to keep 1MB data here?

Component(s)

C++

[felipecrv]

Wait... does Resize to a size smaller than the capacity of the buffer really frees memory?

I wonder if there is something to learn from the zero-copy stream design implemented in protocolbuffers

https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/io/zero_copy_stream.h

[mapleFU]

  Status Resize(const int64_t new_size, bool shrink_to_fit = true) override {
    if (ARROW_PREDICT_FALSE(new_size < 0)) {
      return Status::Invalid("Negative buffer resize: ", new_size);
    }
    uint8_t* ptr = mutable_data();
    if (ptr && shrink_to_fit && new_size <= size_) {
      // Buffer is non-null and is not growing, so shrink to the requested size without
      // excess space.
      int64_t new_capacity = bit_util::RoundUpToMultipleOf64(new_size);
      if (capacity_ != new_capacity) {
        // Buffer hasn't got yet the requested size.
        RETURN_NOT_OK(pool_->Reallocate(capacity_, new_capacity, alignment_, &ptr));
        data_ = ptr;
        capacity_ = new_capacity;
      }
    } else {
      RETURN_NOT_OK(Reserve(new_size));
    }
    size_ = new_size;

    return Status::OK();
  }

@felipecrv Resize has a shrink_to_fit option, wouldn't it free by Reallocate?

[mapleFU]

I wonder if there is something to learn from the zero-copy stream design implemented in protocolbuffers

Do you mean GzipStream here: https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/io/gzip_stream.h ? I think it's logic is similiar ( https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/io/gzip_stream.cc#L97 ). It read from Zero-copy stream, having a 64K internal buffer, arrow uses 1MiB. I'm sure it will keep 64KB memory util end

We might be a bit complex when need_more_output is set. I think we can:

1. Keep 1MiB buffer
2. If larger buffer is need (it could be merely happens), we can allocate a larger temp buffer

[felipecrv]

@felipecrv Resize has a shrink_to_fit option, wouldn't it free by Reallocate?

You're right. The implementation is not a realloc, but a new allocation.
https://github.com/apache/arrow/blob/main/cpp/src/arrow/memory_pool.cc#L353

[pitrou]

You're right. The implementation is not a realloc, but a new allocation. https://github.com/apache/arrow/blob/main/cpp/src/arrow/memory_pool.cc#L353

Late reply to this, but you were looking at the system memory pool. Look for example at the mimalloc memory pool, and you'll see that it reallocates:

arrow/cpp/src/arrow/memory_pool.cc

Lines 419 to 420 in 24feab0

	*ptr = reinterpret_cast<uint8_t*>(
	mi_realloc_aligned(previous_ptr, static_cast<size_t>(new_size), alignment));

Same for the jemalloc memory pool here:

arrow/cpp/src/arrow/memory_pool_jemalloc.cc

Lines 111 to 113 in 24feab0

	*ptr =
	reinterpret_cast<uint8_t*>(rallocx(*ptr, static_cast<size_t>(new_size),
	MALLOCX_ALIGN(static_cast<size_t>(alignment))));