-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix geogrid chunking to accept "auto" and to preserve dtype #61
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #61 +/- ##
==========================================
+ Coverage 88.07% 88.31% +0.24%
==========================================
Files 21 21
Lines 1434 1464 +30
==========================================
+ Hits 1263 1293 +30
Misses 171 171
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
||
vchunks = range(0, shape[0], v_chunk_size) | ||
hchunks = range(0, shape[1], h_chunk_size) | ||
chunks = normalize_chunks(chunks, shape, dtype=self.values.dtype) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there no input dask array to take the chunk size from or to calculate the chunk size from (ex. multiply by a factor)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is very dependent on the input data/reader. In the cases where the chunk sizes are known and provided here, normalize_chunks
should not be interfering.
Eg:
In [6]: normalize_chunks(((2, 2, 2, 2, 2), (2, 2, 2, 2, 2)), (10, 10), dtype=np.float32)
Out[6]: ((2, 2, 2, 2, 2), (2, 2, 2, 2, 2))
However, using it allows passing more abstract values, like 10MiB
or auto
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good
This PR fixes the GeoGridInterpolator to preserve type and accept "auto" and other string chunk specifications.
git diff origin/main **/*py | flake8 --diff