Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IndexRaster for gridding on reduced weighted rasters #10

Merged
merged 25 commits into from
Jun 12, 2024

Conversation

coroa
Copy link
Collaborator

@coroa coroa commented Nov 9, 2023

IndexRaster holds (and is able to serialize) a reduced weighted raster, ie. one that is split in:

  1. a flat indicator, where integer values from 1 to number of shapes indicate cells fully belonging to a single shape
  2. a boundary made up of the weights similar to the previous weighted raster (but only covering cells actually on the boundary)

The boundary uses a MultiIndex called spatial as a sparse representation.

aggregate and grid methods allow to aggregate gridded data and to grid indexed data.

There is a from_weighted_raster constructor which transforms a weighted raster into the reduced form. It would be good to add a rasterization strategy similar to maybe hybrid which directly creates an IndexRaster.

Documentation and testing has not been addressed yet.

Two new dependencies need to be pulled in (we could also make them optional?):

  1. flox is necessary for aggregating data from the grid efficiently
  2. cf_xarray provides two convenience functions which allow serializing and deserializing multi-index coordinates

@coroa
Copy link
Collaborator Author

coroa commented Feb 22, 2024

Ok, was able to fix the failing cf_xarray import by bumping the python version to 3.9, but there are some regression tests failures i am not getting to the bottom of today.

@coroa coroa marked this pull request as ready for review February 22, 2024 16:34
@gidden
Copy link
Owner

gidden commented Feb 29, 2024

Hi @coroa - I will do a more thorough review when I have time later today. But I just ran the tests locally and all pass - so have restarted the job here.

src/ptolemy/raster.py Outdated Show resolved Hide resolved
@gidden
Copy link
Owner

gidden commented Feb 29, 2024

Hi @coroa I have now reviewed and can only admire the approach and implementation. Let me know when you have a chance to update the docs and add tests, which will also help me do a full review.

@gidden
Copy link
Owner

gidden commented Mar 1, 2024

Hey @coroa I just realized here that everything in this PR is a pure addition and no existing code is touched. So I have to say I'm really confused as to why the tests are failing (again noting they pass on my local machine).

@gidden
Copy link
Owner

gidden commented Mar 1, 2024

rebased on main

@codecov-commenter
Copy link

codecov-commenter commented Mar 1, 2024

Codecov Report

Attention: Patch coverage is 38.02817% with 44 lines in your changes are missing coverage. Please review.

Project coverage is 68.25%. Comparing base (3d07c4f) to head (569947c).

Files Patch % Lines
src/ptolemy/raster.py 38.02% 44 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #10      +/-   ##
==========================================
- Coverage   76.73%   68.25%   -8.49%     
==========================================
  Files           2        2              
  Lines         245      315      +70     
  Branches       57       74      +17     
==========================================
+ Hits          188      215      +27     
- Misses         26       69      +43     
  Partials       31       31              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gidden
Copy link
Owner

gidden commented Mar 20, 2024

Current test failures is because of black. Todos left for this PR are:

  1. add tests
  2. change linting to ruff

@coroa
Copy link
Collaborator Author

coroa commented Jun 11, 2024

Ready for review! And merging.

Copy link
Owner

@gidden gidden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great @coroa - I did a scan of the doc notebook and tests, which all make sense to me. Thanks so much for this!

@gidden gidden merged commit d545688 into main Jun 12, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants