Fix garbage collection #2754

akaszynski · 2022-06-08T09:46:36Z

First Problem: Unnecessary Scalar Array Copying

As pointed out, we're copying scalars within add_mesh. When plotting existing scalars with:

import pyvista
mesh = pyvista.Sphere()
mesh['data'] = range(mesh.n_points)
mesh.plot(scalars='data')

We actually create a copy of the scalars:

pyvista/pyvista/plotting/mapper.py

Lines 253 to 263 in 316a110

    
           # Scalars interpolation approach 
        
           if use_points: 
        
               mesh.point_data.set_array(scalars, title, True) 
        
               mesh.active_scalars_name = title 
        
               self.SetScalarModeToUsePointData() 
        
           elif use_cells: 
        
               mesh.cell_data.set_array(scalars, title, True) 
        
               mesh.active_scalars_name = title 
        
               self.SetScalarModeToUseCellData() 
        
           else: 
        
               raise_not_matching(scalars, mesh)

This is inefficient, but this actually protects us from underlying garbage collection issues.

Second Problem: Garbage Collection of VTK Arrays

When performing a shallow copy, we end up with a VTK array memory leak. This can be reproduced with:

import pyvista
mesh = pyvista.Sphere()
mesh.point_data['data'] = range(mesh.n_points)
mesh.point_data['data'] = mesh.point_data['data']

This issue is caused by us recreating the pointer to the underlying VTK array, and the old pointer is not discarded. The solution to this is to simply reuse the underlying pointer.

Requires #2770

Bonus changes

Refactored set_scalars use of configure_scalars_mode as we were calling that twice when setting scalars for custom opacity.

codecov · 2022-06-08T09:55:24Z

Codecov Report

Merging #2754 (cbc95ca) into main (a3d0351) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #2754      +/-   ##
==========================================
- Coverage   93.97%   93.94%   -0.04%     
==========================================
  Files          76       76              
  Lines       16320    16348      +28     
==========================================
+ Hits        15337    15358      +21     
- Misses        983      990       +7

pyvista/core/datasetattributes.py

tests/plotting/test_collection.py

tests/plotting/test_plotting.py

tests/test_utilities.py

pyvista/plotting/mapper.py

Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

pyvista/_vtk.py

pyvista/utilities/helpers.py

tests/test_utilities.py

Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

* Update information of translation project * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update doc/index.rst Co-authored-by: Alex Kaszynski <akascap@gmail.com> Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

Updates the requirements on [hypothesis](https://github.com/HypothesisWorks/hypothesis) to permit the latest version. - [Release notes](https://github.com/HypothesisWorks/hypothesis/releases) - [Commits](HypothesisWorks/hypothesis@hypothesis-ruby-0.0.1...hypothesis-python-6.47.1) --- updated-dependencies: - dependency-name: hypothesis dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: akaszynski <akaszynski@users.noreply.github.com>

* Add generate_polygon arg to Polygon * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: Andras Deak <adeak@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

* skip or avoid creating files in doc build * remove mention of _get_default_tempdir

* Create codeql-analysis.yml * Update README.rst * Update app.py * Fix by isort * Delete codeql-analysis.yml * Apply suggestions from code review

* Add extrude-trim.py * Add example comment * Add test * Add an inplace test * Add extrude_trim filter * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update pyvista/core/filters/poly_data.py Co-authored-by: Phil Chiu <whophil@users.noreply.github.com> * Apply suggestions from code review * Fix by black * Fix by isort * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review Co-authored-by: MatthewFlamm <39341281+MatthewFlamm@users.noreply.github.com> * improve example Co-authored-by: Phil Chiu <whophil@users.noreply.github.com> Co-authored-by: MatthewFlamm <39341281+MatthewFlamm@users.noreply.github.com> Co-authored-by: Alex Kaszynski <akascap@gmail.com>

Updates the requirements on [hypothesis](https://github.com/HypothesisWorks/hypothesis) to permit the latest version. - [Release notes](https://github.com/HypothesisWorks/hypothesis/releases) - [Commits](HypothesisWorks/hypothesis@hypothesis-ruby-0.0.1...hypothesis-python-6.47.2) --- updated-dependencies: - dependency-name: hypothesis dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* optionally fail when image is missing * use in linux tests instead of windows * remove failing test

* fix return types * override __array_wrap__ * Apply suggestions from code review * Update pyvista/core/pyvista_ndarray.py Co-authored-by: Andras Deak <adeak@users.noreply.github.com> Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

akaszynski · 2022-06-14T07:29:07Z

Ready for review @adeak.

pyvista/utilities/helpers.py

pyvista/core/datasetattributes.py

pyvista/utilities/helpers.py

pyvista/core/datasetattributes.py

Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

akaszynski · 2022-06-16T19:34:38Z

@adeak, we're ready for review (again).

adeak

Approving the earlier changes, but @akaszynski please check mine before merging (assuming CI ends up green).

pyvista/utilities/helpers.py

akaszynski added 3 commits June 7, 2022 01:54

do not deep copy on assigning scalars

b169645

fix garbage collection

96e1a59

clean up collection test

2c33e8f

github-actions bot added the bug Uh-oh! Something isn't working as expected. label Jun 8, 2022

akaszynski added 4 commits June 9, 2022 02:04

add types

36099cc

fix docstring

df74f9b

fix docstring

10ec272

fix type

9c78853

akaszynski mentioned this pull request Jun 10, 2022

Do not add arrays when plotting #2763

Merged

adeak reviewed Jun 11, 2022

View reviewed changes

adeak and others added 4 commits June 11, 2022 12:43

Rename shallow copy, fix some typos

2c43a8d

Apply suggestions from code review

872da2c

Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

cleanup based on suggestions

129477c

always return pyvista_ndarray

ea76919

akaszynski mentioned this pull request Jun 11, 2022

Avoid creating a deep copy of point/cell data in Plotter.add_mesh #2469

Closed

Merge branch 'main' into fix/scalar_mapper

4f514e9

adeak reviewed Jun 11, 2022

View reviewed changes

pyvista/_vtk.py Outdated Show resolved Hide resolved

pyvista/utilities/helpers.py Outdated Show resolved Hide resolved

tests/test_utilities.py Outdated Show resolved Hide resolved

akaszynski and others added 12 commits June 12, 2022 08:36

Update tests/test_utilities.py

e00c325

Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

return only pyvista_ndarray

b13da36

fix test

95a9fa3

add missing image cache png (#2765)

462dd74

update pre-commit hooks (#2776)

6659814

Co-authored-by: akaszynski <akaszynski@users.noreply.github.com>

skip or avoid creating files in doc build (#2748)

416d91f

* skip or avoid creating files in doc build * remove mention of _get_default_tempdir

Fix the error running by CodeQL. (#2772)

dde6411

* Create codeql-analysis.yml * Update README.rst * Update app.py * Fix by isort * Delete codeql-analysis.yml * Apply suggestions from code review

MatthewFlamm and others added 4 commits June 14, 2022 01:19

Fail when regression image is missing on CI (#2778)

b0c41a1

* optionally fail when image is missing * use in linux tests instead of windows * remove failing test

fix docstring return types

82f5b4b

Merge branch 'main' into fix/scalar_mapper

a6b046c

adeak reviewed Jun 14, 2022

View reviewed changes

pyvista/utilities/helpers.py Outdated Show resolved Hide resolved

pyvista/utilities/helpers.py Outdated Show resolved Hide resolved

pyvista/utilities/helpers.py Show resolved Hide resolved

akaszynski added 3 commits June 14, 2022 13:30

Merge branch 'main' into fix/scalar_mapper

be67992

cleanup docs

ff94f28

remove code duplication

1d155ca

adeak reviewed Jun 14, 2022

View reviewed changes

adeak and others added 4 commits June 14, 2022 14:17

Minor improvements from code review

018ea89

Merge branch 'main' into fix/scalar_mapper

08a12e8

move complex collection test to test_collection

cfe1109

Apply suggestions from code review

1bfe8af

Co-authored-by: Andras Deak <adeak@users.noreply.github.com>

Fix array retrieval for VTK objects, add tests for coverage

cbc95ca

adeak approved these changes Jun 16, 2022

View reviewed changes

pyvista/utilities/helpers.py Show resolved Hide resolved

akaszynski merged commit ffba268 into main Jun 18, 2022

akaszynski deleted the fix/scalar_mapper branch June 18, 2022 08:53

banesullivan mentioned this pull request Jun 27, 2022

Point data of a shallow-copied mesh is no longer a view after adding the mesh to a plotter #2461

Closed

eudoxos mentioned this pull request Nov 24, 2022

Exception with 0-d np.array dataset attribute #3639

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix garbage collection #2754

Fix garbage collection #2754

akaszynski commented Jun 8, 2022 •

edited

codecov bot commented Jun 8, 2022 •

edited

akaszynski commented Jun 14, 2022

akaszynski commented Jun 16, 2022

adeak left a comment

	# Scalars interpolation approach
	if use_points:
	mesh.point_data.set_array(scalars, title, True)
	mesh.active_scalars_name = title
	self.SetScalarModeToUsePointData()
	elif use_cells:
	mesh.cell_data.set_array(scalars, title, True)
	mesh.active_scalars_name = title
	self.SetScalarModeToUseCellData()
	else:
	raise_not_matching(scalars, mesh)

Fix garbage collection #2754

Fix garbage collection #2754

Conversation

akaszynski commented Jun 8, 2022 • edited

First Problem: Unnecessary Scalar Array Copying

Second Problem: Garbage Collection of VTK Arrays

Bonus changes

codecov bot commented Jun 8, 2022 • edited

Codecov Report

akaszynski commented Jun 14, 2022

akaszynski commented Jun 16, 2022

adeak left a comment

Choose a reason for hiding this comment

akaszynski commented Jun 8, 2022 •

edited

codecov bot commented Jun 8, 2022 •

edited