-
-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor blocked/parallel reprojection #374
Conversation
Codecov Report
@@ Coverage Diff @@
## main #374 +/- ##
==========================================
- Coverage 93.86% 93.37% -0.49%
==========================================
Files 24 25 +1
Lines 847 861 +14
==========================================
+ Hits 795 804 +9
- Misses 52 57 +5
... and 1 file with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
As mentioned in chat we should have a way of parallelising over independent celestial input chunks as well. |
…cify return_type='numpy' or 'dask' for reproject_interp
… mapped arrays on Windows
…n blocked reprojection
@Cadair - I am working on that but will add it in a separate PR. In the mean time, would you be able to review this or are you happy for me to just merge? |
This refactors the handling of blocked/parallel reprojection with dask following a conversation with @Cadair.
return_type
option toreproject_interp
which can be'numpy'
(default) or'dask'
. When the latter is used, nothing is computed whenreproject_interp
is called so this is very fastreturn_type
to'dask'
and then compute it with the desired scheduler (e.g. a dask distributed scheduler). However, we need to make it very clear that the code is not thread-safe, so that multi-threaded schedulers can't be used (I don't think there is a way for us to check that)roundtrip_coords
andorder
- these were not getting passed through to the blocked/parallel reprojection so were getting ignored, but need to add tests to make sure these options do something