Ensure same name arrays do not shallow copy

[akaszynski]

Resolves #2864 by ensuring that same named arrays are not shallow copied, but rather have their reference directly returned by _prepare_array.

For whatever reason, arrays that have been shallow copied still have their underlying arrays collected by VTK despite that they're used.

You can reproduce the memory issue (from a fresh python shell) with:

import numpy as np
import pyvista

points = np.zeros((10000000, 3))
dataset = pyvista.PointSet(points)
data = np.arange(dataset.n_points, dtype=float)

dataset['scalars'] = data.copy()
dataset['scalars'] *= 0.5
print(dataset['scalars'])

[codecov]

Codecov Report

Merging #2872 (268164f) into main (257f4aa) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #2872      +/-   ##
==========================================
- Coverage   94.09%   94.04%   -0.06%     
==========================================
  Files          76       76              
  Lines       16426    16428       +2     
==========================================
- Hits        15456    15449       -7     
- Misses        970      979       +9     

[adeak]

Hmm, I don't think I fully understand the issue, but I guess if the change makes the segfault go away it's definitely the right call.

But looking at the use case of the fix we can come up with an even more absurd test:

import numpy as np
import pyvista

points = np.zeros((10000000, 3))
dataset = pyvista.PointSet(points)
data = np.arange(dataset.n_points, dtype=float)

dataset['scalars'] = data.copy()
dataset['scalars'] = dataset['scalars']
print(dataset['scalars'])

I can't really put my finger on the issue, because it seems to me that in the buggy case execution went like this:

pyvista/pyvista/core/datasetattributes.py

Line 223 in 257f4aa

self.set_array(value, name=key)

pyvista/pyvista/core/datasetattributes.py

Lines 614 to 616 in 257f4aa

	vtk_arr = self._prepare_array(data, name, deep_copy)
	self.VTKObject.AddArray(vtk_arr)
	self.VTKObject.Modified()

And here vtk_arr ended up being a shallow copy instead of a reference:

pyvista/pyvista/core/datasetattributes.py

Lines 775 to 778 in 257f4aa

	vtk_arr = copy_vtk_array(data.VTKObject, deep=deep_copy)
	if isinstance(name, str):
	vtk_arr.SetName(name)
	return vtk_arr

I would assume what's going on is that vtkPointData (through DataSetAttributes) sees that key 'scalars' is being replaced, decreases some internal refcount of the original array, but since it's trying to add the same array back to itself (through the shallow copy) things blow up.

But then why does this not reproduce the segfault:

import numpy as np
import pyvista

points = np.zeros((10000000, 3))
dataset = pyvista.PointSet(points)
data = np.arange(dataset.n_points, dtype=float)
dataset['scalars'] = data.copy()

# pure-pyvsta version
#dataset['scalars'] = dataset['scalars']

# VTK version
shallow = type(dataset['scalars'].VTKObject)()  # create shallow copy
shallow.ShallowCopy(dataset['scalars'].VTKObject)  # initialize shallow copy
shallow.SetName(dataset['scalars'].VTKObject.GetName())  # rename shallow copy
dataset.point_data.VTKObject.AddArray(shallow)  # (re-)add shallow copy
dataset.point_data.VTKObject.Modified()  # set to modified

print(dataset['scalars'])  # no explosion

[akaszynski]

Hmm, I don't think I fully understand the issue, but I guess if the change makes the segfault go away it's definitely the right call.

I spent a while trying every alternative to my proposal, and only this one did the trick.

But then why does this not reproduce the segfault...

I find that perplexing as well, especially as we're effectively following the same steps as in _prepare_array.

Recommending that we proceed and debug as later should this ever crop up again.

[adeak]

Recommending that we proceed and debug as later should this ever crop up again.

Good call, let's be pragmatic here. And let's be suspicious if someone mentions "segfault" in the near future.