Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Raster Datasets tutorial fails without git repo #766

Closed
adamjstewart opened this issue Sep 7, 2022 · 6 comments · Fixed by #772
Closed

Custom Raster Datasets tutorial fails without git repo #766

adamjstewart opened this issue Sep 7, 2022 · 6 comments · Fixed by #772
Labels
datasets Geospatial or benchmark datasets documentation Improvements or additions to documentation
Milestone

Comments

@adamjstewart
Copy link
Collaborator

Description

If you run the Custom Raster Datasets tutorial on Google Colab, it will fail because it requires test data that is not present after a pip install. We should instead either create fake data with rasterio or download real data from somewhere.

P.S. The plot image doesn't show anything. Is this expected? This tutorial doesn't look very useful as is.

@RitwikGupta

Steps to reproduce

Run the tutorial on Colab. The cell that tries to unzip the test data fails with the following error message:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-3-304b17c7ea53>](https://localhost:8080/#) in <module>
      2 
      3 data_root = Path("../../tests/data/xview3/")
----> 4 extract_archive(str(data_root / "sample_data.tar.gz"))

3 frames
[/usr/lib/python3.7/gzip.py](https://localhost:8080/#) in __init__(self, filename, mode, compresslevel, fileobj, mtime)
    166             mode += 'b'
    167         if fileobj is None:
--> 168             fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
    169         if filename is None:
    170             filename = getattr(fileobj, 'name', '')

FileNotFoundError: [Errno 2] No such file or directory: '../../tests/data/xview3/sample_data.tar.gz'

Version

releases/v0.3

@adamjstewart adamjstewart added documentation Improvements or additions to documentation datasets Geospatial or benchmark datasets labels Sep 7, 2022
@adamjstewart adamjstewart added this to the 0.3.1 milestone Sep 7, 2022
@adamjstewart
Copy link
Collaborator Author

Going to replace the xView3 example with a more complex example that downloads real data (maybe Sentinel), overrides both filename_glob and filename_regex, and plots an RGB image.

@RitwikGupta
Copy link
Collaborator

@adamjstewart That sounds good to me. This was a contribution with many downfalls, such as displaying an all-white raster (hence why the plot image doesn't show "anything") and relying on a test file that has to be part of the repo itself.

@calebrob6
Copy link
Member

@RitwikGupta, do you have a xView3 mini dataset that could be published on Zenodo?

@RitwikGupta
Copy link
Collaborator

@calebrob6 there is an xView3 tiny dataset, but there are two issues:

  • The tiny dataset is comprised of 9 scenes, but each scene still has two files, VV_dB.tif and VH_dB.tif that are ~1.2 GB each.
  • The xView3 T&Cs prevent rehosting the dataset on other services. This is due to a requirement to track access of the data.

@Sandipriz
Copy link

Why RasterDataset in colab throws an error?
root = Path('dset-s2')
assert root.exists()

train_imgs = RasterDataset(root=root/'tra_scene', transforms=scale_image, crs='epsg:3395', res=10)

TypeError: RasterDataset.init() got an unexpected keyword argument 'root

@adamjstewart
Copy link
Collaborator Author

@Sandipriz the root argument was renamed to paths in TorchGeo 0.5, see https://torchgeo.readthedocs.io/en/stable/api/datasets.html#torchgeo.datasets.RasterDataset. Sorry for any confusion!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datasets Geospatial or benchmark datasets documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants