-
Notifications
You must be signed in to change notification settings - Fork 304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add L8 Biome Dataset #1200
Add L8 Biome Dataset #1200
Conversation
CDL is useless for
First of all, cmap isn't required for any dataset. But you do need a way to plot the image, and imshow + colormap is the best way to do this. Oftentimes, like in the case of CDL, this cmap is hardcoded into the GeoTIFF file, and TorchGeo will automatically read and use it without listing it in the dataset, but I don't think that's the case with your masks. In the case of this dataset, the authors purposefully chose 0, 64, 128, 192, 255 for the class values instead of 0, 1, 2, 3, 4. I'm guessing that's because these are the cmap values they recommend. Basically, we plot the image in grayscale with each class having these color values. Note that in the |
For the failing tests, see https://torchgeo.readthedocs.io/en/stable/user/contributing.html#linters |
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
…orchgeo into datasets/l8biome
Latest modifications:
Doubts:
TODOs:
Tagging @adamjstewart for review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is very close to being done (aside from plotting and testing)!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hopefully my last review!
Docs look good: https://torchgeo--1200.org.readthedocs.build/en/1200/api/datasets.html#l8-biome
Can you plot an image and add the plot to the PR description so we can make sure everything looks visually correct? Colors might be better now that it isn't only B2 data.
Downloading raw data directly Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great, congrats on your first PR! Hopefully the first of many if the extensive review process didn't scare you away 😅
* Added varification of data * Added getitem and plotting functions - partial * Update torchgeo/datasets/l8biome.py Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * Changes corresponding to draft PR * Changed auxilliary files for dataset addition; added L8Biome datamodule * Fixed dataset verification and download; minor changes in variable declaration * Running linters to properly format code * Fixing failed linter tests from previous commit * Fixing comments from review microsoft#2 * Added script to generate artificial data in similar format as original files * Removed dependency on existing dataset * Fixes to the getitem function; tested with samples * Making suggested changes; fixing mask; writing tests * Changes to plot RGB image * Changes to plot RGB image - running into errors related to band variables declaration * Fixed plotting of dataset * Added masks to artificial data generation; tested with pytest - all tests pass * Resolving file and code formatting with linters * Added artificial data * Fixing lists in docstrings * Fixing comments from review * Added new tests * Regenerated artificial data * Added test for l8datamodule; linters formatting * Fixed image generation with more rigorous check on filepath * docstring formatting change * Resolving CI test errors * Added conf/yaml for datamodules test * Fixing CI test failure bug; Changes from review; Samplers in datamodule WIP * Added more artificial data * Fixed yaml and datamodule changes * Running linters to properly format code * Reducing batch_size and length for datamodules tests * Reduced patch size in l8biome yaml to fit within artificial data * Declaring all samplers each time * Reverting to stages checks for samplers * Changed data generation file and generated data again * Fix getitem * Included more encompassing tests * Adding test for RGB bands being absent; running linters * Added check for mask in getitem test * Changes corresponding to review * Update torchgeo/datasets/l8biome.py Downloading raw data directly Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * Update l8biome.py * Update torchgeo/datasets/l8biome.py: Changing RGB bands order Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com> * Change B2 band to B1 --------- Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Overview
Adds L8 Biome dataset as a RasterDataset to TorchGeo.
Main changes:
Here is a sample plot of an image and its mask from the dataset: