Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for temporal data #1803

Merged
merged 11 commits into from Jun 23, 2023
Merged

Add support for temporal data #1803

merged 11 commits into from Jun 23, 2023

Conversation

AdeelH
Copy link
Collaborator

@AdeelH AdeelH commented Jun 20, 2023

Overview

This PR allows Raster Vision to read temporal data i.e. sequences of images.

Major changes:

  • New TemporalMultiRasterSource that stacks (rather than concatenates (like MultiRasterSource)) images from sub sources to get spatiotemporal chips of shape (T, H, W, C).
  • XarraySource can now accept and read from a DataArray with a time dimension if temporal=True.
  • MultiRasterSource can now work with sub raster sources that return >3D chips.
  • RasterTransformers can now work with >3D chips.
  • RasterStats can now work with >3D chips.
  • AlbumentationDataset (base class of GeoDatasets) can now read from a dataset that returns 4D chips.
  • Visualizers can now plot batches of 4D chips.
  • New demo notebook: Working with time-series of images

Checklist

  • Added needs-backport label if PR is bug fix that applies to previous minor release
  • Ran scripts/format_code and committed any changes
  • Documentation updated if needed
  • PR has a name that won't get you publicly shamed for vagueness

Notes

Even though RV GeoDatasets can now return temporal data, training a model on this kind with the default Learners will not work because the default models are image-level models. Therefore, the user will need to provide custom models that work with sequential data.

Testing Instructions

  • See updated/new unit tests and new demo notebook

Closes #1802

@AdeelH AdeelH force-pushed the temporal branch 3 times, most recently from 7e113b6 to f2476b1 Compare June 22, 2023 19:16
- MultiRasterSourceConfig can now also build a TemporalMultiRasterSource if temporal=True
- MultiRasterSource can now work with temporal sub raster sources
@codecov
Copy link

codecov bot commented Jun 23, 2023

Codecov Report

Merging #1803 (0d5761e) into master (ba2dc8a) will increase coverage by 0.29%.
The diff coverage is 91.83%.

@@            Coverage Diff             @@
##           master    #1803      +/-   ##
==========================================
+ Coverage   79.53%   79.83%   +0.29%     
==========================================
  Files         187      188       +1     
  Lines        9038     9182     +144     
==========================================
+ Hits         7188     7330     +142     
- Misses       1850     1852       +2     
Impacted Files Coverage Δ
...astervision/core/data/label_source/label_source.py 75.00% <50.00%> (ø)
...data/label_source/object_detection_label_source.py 83.63% <50.00%> (ø)
.../core/data/raster_transformer/stats_transformer.py 77.14% <50.00%> (ø)
...rvision/core/data/raster_source/rasterio_source.py 79.85% <81.81%> (-0.43%) ⬇️
...ervision_core/rastervision/core/data/utils/misc.py 81.55% <88.23%> (-1.28%) ⬇️
...data/raster_source/temporal_multi_raster_source.py 89.36% <89.36%> (ø)
.../rastervision/pytorch_learner/dataset/transform.py 95.12% <90.47%> (+0.67%) ⬆️
...tervision/core/data/raster_source/raster_source.py 91.04% <92.30%> (+7.17%) ⬆️
...tervision/core/data/raster_source/xarray_source.py 92.23% <93.54%> (-0.54%) ⬇️
...astervision_core/rastervision/core/raster_stats.py 98.75% <94.11%> (-1.25%) ⬇️
... and 13 more

... and 1 file with indirect coverage changes

@AdeelH AdeelH marked this pull request as ready for review June 23, 2023 20:40
@AdeelH AdeelH merged commit a38f46a into azavea:master Jun 23, 2023
2 checks passed
@AdeelH AdeelH deleted the temporal branch June 23, 2023 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support reading temporal data (i.e. time-series of multiple images of the same scene)
1 participant