Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce memory use, improve pixel classification measurement performance #1332

Merged
merged 7 commits into from Sep 15, 2023
Merged

Reduce memory use, improve pixel classification measurement performance #1332

merged 7 commits into from Sep 15, 2023

Conversation

petebankhead
Copy link
Member

Numerous changes aiming to reduce memory use, including:

  • Preventing objects (and object hierarchies) from being retained for a time after an image has been closed
  • Improving the efficiency of ImageOps with padding
  • Parallelising pixel classification measurements

The last two of these should improve pixel classification generally, and making measurements in particular.

It is the proposed fix for #1322

I obtained some rough timings using Run for project with 3 images and an extreme pixel classifier with 250 output channels:

setImageType('BRIGHTFIELD_H_E');
setColorDeconvolutionStains('{"Name" : "H&E default", "Stain 1" : "Hematoxylin", "Values 1" : "0.65111 0.70119 0.29049", "Stain 2" : "Eosin", "Values 2" : "0.2159 0.8012 0.5581", "Background" : " 255 255 255"}');
clearAllObjects();
resetSelection();
//createAnnotationsFromPixelClassifier("My threshold", 0.0, 0.0, "INCLUDE_IGNORED", "SPLIT") // Alternative option
createAnnotationsFromPixelClassifier("My threshold", 0.0, 0.0, "INCLUDE_IGNORED")
selectAnnotations();
addPixelClassifierMeasurements("Busy classifier", "Busy classifier")
println getAnnotationObjects().measurements

With larger images / ROIs, the changes can result in huge speed improvements (tested using M1 Max, macOS 13.5.2):

v0.5.0-SNAPSHOT

INFO: Starting script at Fri Sep 15 13:54:32 BST 2023
INFO: [[Busy classifier: Stroma %:57.11750411987305, Busy classifier: Stroma area µm^2:3.7282948E7, Busy classifier: Tumor %:42.88249588012695, Busy classifier: Tumor area µm^2:2.799117E7]]
INFO: Total run time: 14.97 seconds
INFO: Starting script at Fri Sep 15 13:54:47 BST 2023
INFO: [[Busy classifier: Stroma %:24.72903823852539, Busy classifier: Stroma area µm^2:122532.3984375, Busy classifier: Tumor %:75.27095794677734, Busy classifier: Tumor area µm^2:372967.65625]]
INFO: Total run time: 0.42 seconds
INFO: Starting script at Fri Sep 15 13:54:48 BST 2023
INFO: [[Busy classifier: Stroma %:42.06702423095703, Busy classifier: Stroma area µm^2:2.7533646E7, Busy classifier: Tumor %:57.93297576904297, Busy classifier: Tumor area µm^2:3.7918208E7]]
INFO: Total run time: 9.97 seconds

v0.4.4

INFO: Starting script at Fri Sep 15 13:57:43 BST 2023
INFO: [[Busy classifier: Stroma %:57.11750411987305, Busy classifier: Stroma area µm^2:3.7282948E7, Busy classifier: Tumor %:42.88249588012695, Busy classifier: Tumor area µm^2:2.799117E7]]
INFO: Total run time: 78.18 seconds
INFO: Starting script at Fri Sep 15 13:59:01 BST 2023
INFO: [[Busy classifier: Stroma %:24.72903823852539, Busy classifier: Stroma area µm^2:122532.3984375, Busy classifier: Tumor %:75.27095794677734, Busy classifier: Tumor area µm^2:372967.65625]]
INFO: Total run time: 0.49 seconds
INFO: Starting script at Fri Sep 15 13:59:02 BST 2023
INFO: [[Busy classifier: Stroma %:42.06702423095703, Busy classifier: Stroma area µm^2:2.7533646E7, Busy classifier: Tumor %:57.93297576904297, Busy classifier: Tumor area µm^2:3.7918208E7]]
INFO: Total run time: 74.90 seconds

@petebankhead
Simplify (slightly) the drawing tool handlers
fca9352 
The aim is to avoid a memory leak where storing a 'parent object' when drawing (in order to constrain some tools, such as the brush and wand) would inadvertently result in a whole object hierarchy remaining in memory.

This could cause hundreds of MB to retained unnecessarily after an image was closed.

In practice, the issue was likely to be resolved quickly through interacting with the viewer some more, but it would be better to avoid the problem entirely.
@petebankhead
Update AnnotationPane.java
4fad137 
Ensure we do not retain a reference to the object hierarchy when the image is null.
@petebankhead
Avoid retaining objects when an image is closed
a2fba10 
Avoid retaining entire object hierarchies within the `ViewerManager`, drawing tools, or `HierarchyOverlay` after images have been closed.
@petebankhead
Update AbstractImageRegionStore.java
1f391bc 
Ensure that the list of `TileRequestCollection` can have its final item removed, to avoid accidentally holding on to an `ImageServer` reference for too long.
@petebankhead
Improve ImageOps/pixel classification performance
9d80983 
Reduce memory use & computation in two main ways:
- Strip padding in `SplitMergeOp` *before* calculation, rather than after
  - This helps when the split/merge uses different amounts of padding
- Strip padding before merging channels, rather than after

Together, these can result in fewer pixels being processed, and less need to generate multiple large arrays to merge channels.

In one extreme example (fairly high-resolution pixel classifier using all RGB features across multiple scales) I saw a >20% reduction in computation time.
@petebankhead
Update LoadResourceCommand.java
56921c8 
Reinstate live measurements when loading a pixel classifier (not just training one).
@petebankhead
Improve pixel classification measurement performance
ea19aa9 
Parallelize making measurements for pixel classifiers, both at the object level and the tile level.
This can give substantial performance improvements when measuring across large or multiple objects.

Introduce `PixelClassificationMeasurementManager.addMeasurements(Collection, String)` method to move the main logic inside the class (not in `PixelClassifierTools`).
@petebankhead petebankhead added this to the v0.5.0 milestone Sep 15, 2023
@petebankhead petebankhead merged commit 4d12de5 into qupath:main Sep 15, 2023
3 checks passed
@petebankhead petebankhead mentioned this pull request Sep 16, 2023
Closed
@petebankhead petebankhead deleted the leaks branch October 27, 2023 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v0.5.0-rc1
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@petebankhead