Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve interpolation performance #358

Merged
merged 5 commits into from
Mar 18, 2024
Merged

Conversation

rettigl
Copy link
Member

@rettigl rettigl commented Mar 12, 2024

replace scipy.interpolate.RegularGridInterpolator by scipy.ndimage.map_coordinates for a substantial improvement

of binning performance with momentum correction

Closes #357

…p_coordinates for a substantial improvement

of binning performance with momentum correction
@rettigl rettigl changed the title Improve interpolation performacne Improve interpolation performance Mar 12, 2024
@coveralls
Copy link
Collaborator

coveralls commented Mar 12, 2024

Pull Request Test Coverage Report for Build 8286464547

Details

  • 23 of 31 (74.19%) changed or added relevant lines in 1 file are covered.
  • 5 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.1%) to 91.369%

Changes Missing Coverage Covered Lines Changed/Added Lines %
sed/calibrator/momentum.py 23 31 74.19%
Files with Coverage Reduction New Missed Lines %
sed/calibrator/momentum.py 5 86.08%
Totals Coverage Status
Change from base Build 8230697770: -0.1%
Covered Lines: 5992
Relevant Lines: 6558

💛 - Coveralls

@rettigl
Copy link
Member Author

rettigl commented Mar 15, 2024

I tested this again on the maxwell jhub, and there the speedup is really tremendous. Old version is ~70-80 s per iteration, new is 10:
grafik
And indeed, both produce identical results.

Copy link
Member

@zain-sohail zain-sohail left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't quite clear to me why the map_coordinates is faster than scipy's interpolate method. But of course, those are seperate modules and seem to have very different implementations.

The parallelization is a nice addition. Everything seems good

@rettigl
Copy link
Member Author

rettigl commented Mar 18, 2024

It isn't quite clear to me why the map_coordinates is faster than scipy's interpolate method. But of course, those are seperate modules and seem to have very different implementations.

The parallelization is a nice addition. Everything seems good

For comparison of different methods see e. g. here: https://stackoverflow.com/questions/53857403/scipy-interpolation-speed-is-there-anything-faster-than-rectbivariatespline

@rettigl rettigl merged commit 8c0e91b into main Mar 18, 2024
6 checks passed
@rettigl rettigl deleted the interpolation_performance_fix branch March 18, 2024 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Binning extremely slow with current poetry versions
3 participants