GH-36614: [MATLAB] Subclass arrow::Buffer to keep MATLAB data backing arrow::Arrays alive

[sgilmore10]

Rationale for this change

When building arrow.array.<Numeric>Arrays from native MATLAB arrays, we avoid copying by

1. Wrapping the MATLAB data inside a non-owning arrow::Buffer
2. Constructing an arrow::ArrayData object from the arrow::Buffer
3. Constructing the arrow::Array from the arrow::Data object.

Because the Array's underlying Buffer does not have ownership of its backing data, we have been storing the original MATLAB array as a property called MatlabArray on the MATLAB arrow.array.NumericArray class.

This solution is not ideal because the backing MATLAB array is kept separate from the actual std::shared_ptr<arrow::Array>, which is stored within the C++ proxy objects (e.g. arrow::matlab::array::proxy::NumericArray<float64>). A better solution would be to create a new subclass of arrow::Buffer called MatlabBuffer, which will keep the original MATLAB array alive by taking it in as an input parameter and storing it as a member variable.

What changes are included in this PR?

1. Removed the MatlabArray property from the MATLAB class matlab.io.NumericArray
2. Added a private member variable called mda_array to arrow::matlab::array::proxy::NumericArray<CType> and arrow::matlab::array::proxy::TimestampArray. mda_array is a matlab::data::Array object. This member variable stores the MATLAB array that owns the memory the arrow::Array objects are backed by.

Are these changes tested?

Existing tests used.

Are there any user-facing changes?

No.

• Closes: [MATLAB] Subclass arrow::Buffer to keep MATLAB data backing arrow::Arrays alive  #36614

[pitrou]

From the POV of Arrow C++, the idiomatic way to solve this issue would be to define your own Buffer subclass that keeps the required MATLAB object(s) alive. This is what we already do in PyArrow when we want to view a Numpy array as a PyArrow array:

arrow/python/pyarrow/src/arrow/python/numpy_convert.h

Lines 42 to 49 in c09b8c8

	class ARROW_PYTHON_EXPORT NumPyBuffer : public Buffer {
	public:
	explicit NumPyBuffer(PyObject* arr);
	virtual ~NumPyBuffer();
	private:
	PyObject* arr_;
	};

arrow/python/pyarrow/src/arrow/python/numpy_convert.cc

Lines 40 to 58 in c09b8c8

	NumPyBuffer::NumPyBuffer(PyObject* ao) : Buffer(nullptr, 0) {
	PyAcquireGIL lock;
	arr_ = ao;
	Py_INCREF(ao);
	if (PyArray_Check(ao)) {
	PyArrayObject* ndarray = reinterpret_cast<PyArrayObject*>(ao);
	auto ptr = reinterpret_cast<uint8_t*>(PyArray_DATA(ndarray));
	data_ = const_cast<const uint8_t*>(ptr);
	size_ = PyArray_SIZE(ndarray) * PyArray_DESCR(ndarray)->elsize;
	capacity_ = size_;
	is_mutable_ = !!(PyArray_FLAGS(ndarray) & NPY_ARRAY_WRITEABLE);
	}
	}
	NumPyBuffer::~NumPyBuffer() {
	PyAcquireGIL lock;
	Py_XDECREF(arr_);
	}

[sgilmore10]

Thanks for the suggestion @pitrou! That is definitely a better approach.

[sgilmore10]

Based on @pitrou suggestion, I defined a Buffer subclass called MatlabBuffer that keeps the MATLAB data alive. Thanks again for the feedback!

[kou]

+1

Could you update the PR title/description before we merge this?

[sgilmore10]

+1

Could you update the PR title/description before we merge this?

Updated!

[kevingurney]

+1

Thanks for making this change, @sgilmore10!

[kevingurney]

Perhaps we should go with the convention of naming these input Array subclass constructor arguments array for consistency across the various subclasses.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit d8a3360.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about possible false positives for unstable benchmarks that are known to sometimes produce them.