Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FAIR1M Dataset #232

Merged
merged 9 commits into from
Dec 11, 2021
Merged

Add FAIR1M Dataset #232

merged 9 commits into from
Dec 11, 2021

Conversation

isaaccorley
Copy link
Collaborator

@isaaccorley isaaccorley commented Nov 10, 2021

This PR adds the FAIR1M dataset which is listed as an ISPRS benchmark dataset and currently being used in the Gaofen Challenge. The dataset is hosted on Google Drive but is private and requires sign in to the Gaofen Challenge to download so we aren't able to support automatic downloads.

The dataset is also split into several parts released throughout the challenge duration. I haven't received an email yet with the updated links so I'm only including part 1 for now.

  • Adds torchgeo.datasets.FAIR1M and torchgeo.datasets.FAIR1MDataModule

Notes:

  • The objects are oriented/rotated so the targets are actually 5 point polygons so they aren't true bounding boxes. This is different than what we have encountered in other datasets but we should definitely consider supporting this. I've seen this represented as either just 4 point bounding boxes or a non rotated bounding box with an associated rotation angle.

TODO:

  • add methods for verifying checksum and directories
    - add method for plotting samples
  • add part2 of dataset

Examples using the FAIR1M.plot method:

image

image

image

@isaaccorley isaaccorley added the datasets Geospatial or benchmark datasets label Nov 10, 2021
@isaaccorley isaaccorley self-assigned this Nov 10, 2021
@isaaccorley isaaccorley marked this pull request as draft November 10, 2021 03:56
@adamjstewart adamjstewart added this to the 0.2.0 milestone Nov 20, 2021
@isaaccorley isaaccorley marked this pull request as ready for review December 10, 2021 04:15
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
@isaaccorley isaaccorley merged commit 14c19e3 into microsoft:main Dec 11, 2021
@isaaccorley isaaccorley deleted the datasets/fair1m branch December 11, 2021 00:09
@adamjstewart adamjstewart added datamodules PyTorch Lightning datamodules utilities Utilities for working with geospatial data and removed utilities Utilities for working with geospatial data labels Jan 2, 2022
@adamjstewart
Copy link
Collaborator

Tried testing this datamodule with our new ObjectDetectionTask but it doesn't seem compatible. Looking at the fake data, it seems like all boxes are unrotated, but looking at the plot in this PR, it seems like they are rotated in general. Am I correct that we would need #840 in order to properly train this datamodule?

@isaaccorley
Copy link
Collaborator Author

Tried testing this datamodule with our new ObjectDetectionTask but it doesn't seem compatible. Looking at the fake data, it seems like all boxes are unrotated, but looking at the plot in this PR, it seems like they are rotated in general. Am I correct that we would need #840 in order to properly train this datamodule?

That's correct. We don't currently support oriented object detection. Right now we simply provide a way to load the data but not train on it.

@austinmw
Copy link

@isaaccorley How can I download the dataset now that the challenge page is down?

@isaaccorley
Copy link
Collaborator Author

@austinmw Not sure. I can reach out to the authors but I do have a subset of it hosted here

@austinmw
Copy link

Thanks! Very interested to obtain it.

@isaaccorley
Copy link
Collaborator Author

@austinmw I'm able to access the google drive link from here after logging in. Can you check if it works for you?

@isaaccorley
Copy link
Collaborator Author

isaaccorley commented Apr 22, 2023

@austinmw Here's a quick and dirty bash script if you want to automatically download the entire FAIR1M 2.0 dataset using the gdown library

@austinmw
Copy link

Thanks a ton!

@adamjstewart
Copy link
Collaborator

@isaaccorley can you update the dataset download instructions?

@isaaccorley
Copy link
Collaborator Author

@adamjstewart I'm actually working on a PR to update the entire dataset.

yichiac pushed a commit to yichiac/torchgeo that referenced this pull request Apr 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datamodules PyTorch Lightning datamodules datasets Geospatial or benchmark datasets
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants