[Bug]: Large image draw performance deterioration in recent releases #27554
Comments
|
I know you stated that
But I just want to double check whether you have tried the "nearest" interpolation method, as that was the default prior to v3.2 |
|
I have tried both and it does not appear to make a material difference for me. |
|
Can you make a reproducible example that doesn't rely on having tk installed? |
|
As requested below. It actually appears that _rgb_to_rgba may be the culprit |
|
As a further reference point, executing the same code with matplotlib 2.2.2 gives the results below: So, on my machine: |
|
This is still hard for me to debug because it has a mouse click, which zooms on my machine, changing the size of the canvas. Does this require the interactive component? |
|
Here is a version that is not interactive. For me this takes about 1 second to execute on version 2.2.2 and about 2.7 seconds on version 3.8.2. |
|
There are two calls to it in The comments seem to indicate this is because the C++ extension needs it that way. But the first call to So if I've gone through all cases correctly, there is no Python code that needs to call the resampler with RGBA data instead of RGB; it's just that way because it used to be that way? |
|
We should be able to drop the first call with: diff --git a/lib/matplotlib/image.py b/lib/matplotlib/image.py
index 8b672e5ad2..249f58bcb2 100644
--- a/lib/matplotlib/image.py
+++ b/lib/matplotlib/image.py
@@ -555,11 +555,14 @@ class _ImageBase(martist.Artist, cm.ScalarMappable):
if A.ndim == 2: # _interpolation_stage == 'rgba'
self.norm.autoscale_None(A)
A = self.to_rgba(A)
- if A.shape[2] == 3:
- A = _rgb_to_rgba(A)
alpha = self._get_scalar_alpha()
- output_alpha = _resample( # resample alpha channel
- self, A[..., 3], out_shape, t, alpha=alpha)
+ if A.shape[2] == 3:
+ # No need to resample alpha or make a full array; NumPy will expand
+ # this out when it's assigned to the alpha channel below.
+ output_alpha = 255 if A.dtype == np.uint8 else 1.0
+ else:
+ output_alpha = _resample( # resample alpha channel
+ self, A[..., 3], out_shape, t, alpha=alpha)
output = _resample( # resample rgb channels
self, _rgb_to_rgba(A[..., :3]), out_shape, t, alpha=alpha)
output[..., 3] = output_alpha # recombine rgb and alphawhich cuts the time in half again. Removing the second one would require some spelunking into the image extension, though I'm not sure it's possible as it also cuts in Agg as well. |
|
@QuLogic Thanks for working on this. Your patch gives me the following error: |
|
Sorry. Retract that. My error in applying the diff. That does indeed help |
|
However, even with that patch, version 3.8.2 is, still 2.5 times slower than 2.2.2. |
|
FWIW, The second major time consumer seems to be Anyway, for 3.9, the wrapper code is moved to pybind11, which hopefully should not degrade the performance further if nothing else... |
|
@oscasgus I had similarly compared the code bases and did not see major differences. I have also done some testing with NumPy 1.19 vs 1.26, which are the relevant versions for my installs, and could not uncover a cause there either. I am little suspicious that it may have to do with memory allocation. I have profiled using memory_profiler for 2.2.2 and 3.8.2 as shown in these images Here is the line level trace for 2.2.2 And the line level trace for 3.8,2 It appears to me that with version 2.2.2, more memory is allocated to the image artist before the draw is called and that this memory is released after the draw completes. With 3.8.2 it appears that the artist has less memory allocated before the draw and that it has to allocate more during the draw process, which therefore takes more time. Hoping this might provide some additional clues? |
|
I am pursuing threads that I don't really understand, so I hope this is not distracting from the core issue, but one thing that interests me is that in Issue 4567 it is suggested that no copy of the image data is made and yet in image.py at line 686 a call is made to safe_masked_invalid that overrides copy=False to make a copy. While this seems like this may be an issue, even removing this check completely does not seem to have any effect on the issue I reported here, but I thought I should raise it in case it sheds any light on the matter. |
|
A quick note about memory. IIRC, a decision at one point was made to move
from float64 to float32 for some internal image processing operations and
data representation. Mainly because matplotlib did not need that much color
depth representation.
…On Fri, Dec 22, 2023 at 11:20 AM ilopata1 ***@***.***> wrote:
I am pursuing threads that I don't really understand, so I hope this is
not distracting from the core issue, but one thing that interests me is
that in Issue 4567 <#4567>
it is suggested that no copy of the image data is made and yet in image.py
at line 686
<https://github.com/matplotlib/matplotlib/blob/e5a85f960b2d47eac371cff709b830d52c36d267/lib/matplotlib/image.py#L686>
a call is made to safe_masked_invalid that overrides copy=False to make a
copy.
While this seems like this may be an issue, even removing this check
completely does not seem to have any effect on the issue I reported here,
but I thought I should raise it in case it sheds any light on the matter.
—
Reply to this email directly, view it on GitHub
<#27554 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACHF6BENKOLZ265RPJDNBDYKWXL7AVCNFSM6AAAAABA6QIQXGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRXHA3DGNBSG4>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
Thanks a lot for the reproducible example. If I change the call to If I remove the song and dance with alpha from matplotlib/lib/matplotlib/image.py Lines 560 to 565 in e5a85f9 then the speed is much faster at 1.4 s or so, which is what I get for 2.2.2. So I think QuLogic's idea above is sound, and thats probably the best we can do. |
|
Thank you all for you interest, help and support. I think we can consider this closed. My only outstanding question is whether QuLogic's code might get merged or whether I will need to maintain this as a patch for my own use? |
This is probably the best we can do to fix matplotlib#27554 at this time, without delving into re-writing the Agg resampler (which, as the Python code notes, does require RGBA.)
This is probably the best we can do to fix matplotlib#27554 at this time, without delving into re-writing the Agg resampler (which, as the Python code notes, does require RGBA.)
This is probably the best we can do to fix matplotlib#27554 at this time, without delving into re-writing the Agg resampler (which, as the Python code notes, does require RGBA.)
Bug summary
Draw performance for large images has deteriorated significantly between Matplotlib 3.1 and 3.8. The biggest issue seems to arise when upgrading from 3.1. to 3.2
Code for reproduction
Actual outcome
When running Matplotlib 3.8.2
When running Matplotlib 3.1.3
Expected outcome
I would hope that performance remained reasonable consistent or improved between versions.
Additional information
I have only been able to trace this as far as _make_image. The issue seems to persist regardless of interpolation method.
Operating system
Windows 11 64-bit
Matplotlib Version
3.2 upward
Matplotlib Backend
TkAgg
Python version
3.6 for Matplotlib 3.1.3; 3.9 for Matplotlib 3.8.2
Jupyter version
n/A
Installation
conda
The text was updated successfully, but these errors were encountered: