Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix performance regression in base resampler class when comparing geometries #520

Merged
merged 1 commit into from
Jun 8, 2023

Conversation

djhoese
Copy link
Member

@djhoese djhoese commented May 31, 2023

As mentioned in #517, the new comparison added to the base resampler class in #508 was significantly slowing down any cases where non-areas were involved. This was most noticeable with EWA resampling as it is based on this resampler class and only deals with swath definitions.

The primary issue is/was that if the swaths are not equal the == used in the if statement was comparing the every coordinate (every pixel) of the source swath arrays to every pixel of the area definitions generated coordinates. This is usually not necessary since when we get to that point the objects are probably not equal and two swaths that aren't the same exact objects (or the underlying arrays are not the same) are probably not equal even if they have the same shape.

This PR is the first step in a proper fix. This should be released in a 1.27.1 release to fix the performance issue. The second step will be a separate PR (as discussed on the pytroll slack) where a coordinates_equal method will be added to the geometries to compare coordinates values no matter the cost in performance. The __eq__ method will be updated to be strict about objects needing to be the same type and then take the shortcuts implemented in this PR.

@djhoese djhoese added bug performance improves speed or decreases memory consumption, but does not otherwise change functionality labels May 31, 2023
@djhoese djhoese requested a review from mraspaud May 31, 2023 16:08
@djhoese djhoese self-assigned this May 31, 2023
@codecov
Copy link

codecov bot commented May 31, 2023

Codecov Report

Merging #520 (9e6101a) into main (2b94b70) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #520      +/-   ##
==========================================
+ Coverage   94.32%   94.33%   +0.01%     
==========================================
  Files          79       79              
  Lines       12944    12976      +32     
==========================================
+ Hits        12209    12241      +32     
  Misses        735      735              
Flag Coverage Δ
unittests 94.33% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pyresample/test/test_resamplers/test_resampler.py 100.00% <100.00%> (ø)

@coveralls
Copy link

Coverage Status

Coverage: 93.936% (+0.09%) from 93.846% when pulling 9e6101a on djhoese:regression-swath-comparison into 2b94b70 on pytroll:main.

@djhoese djhoese merged commit 7a434c1 into pytroll:main Jun 8, 2023
@djhoese djhoese deleted the regression-swath-comparison branch June 8, 2023 14:42
@djhoese
Copy link
Member Author

djhoese commented Jun 8, 2023

Oh shoot. I thought this already had approval and I just forgot to merge it. Sorry @mraspaud. Let me know if you want any changes to this. I completely didn't realize this wasn't already reviewed.

@djhoese
Copy link
Member Author

djhoese commented Jun 8, 2023

Note the user in the related issue said that this PR fixes their performance issues.

@mraspaud
Copy link
Member

mraspaud commented Jun 9, 2023

No worries, LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug performance improves speed or decreases memory consumption, but does not otherwise change functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EWA resampling in 1.27 slows down four times than 1.26.1
3 participants