Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Drive export and import method #26

Closed
aazuspan opened this issue Sep 20, 2021 · 4 comments
Closed

Add Drive export and import method #26

aazuspan opened this issue Sep 20, 2021 · 4 comments
Labels
enhancement New feature or request

Comments

@aazuspan
Copy link
Owner

This would add two methods allowing ee.ImageCollection and its subclass objects to be exported to a Drive and then imported into an xarray.Dataset. Dimension and coordinates would be stored in filenames and parsed on import. This feature would allow users to handle time series data when file size or grid size is too large or computations time out.

Planned usage reference:

ts = wxee.TimeSeries("IDAHO_EPSCOR/GRIDMET").filterDate("2020", "2021")
task = ts.wx.to_drive(crs="EPSG:5070", scale=4_000)

# Once files are exported, user manually downloads them to a local folder
data_dir = "data"

ds = wxee.load_dataset(data_dir)

Drive exporting will be very similar to the wxee.image._get_url method but will instead run and return a batch export task. All of the importing functionality is already implemented in the private wxee.utils._dataset_from_files, so that portion should be simple.

@aazuspan aazuspan added the enhancement New feature or request label Sep 20, 2021
@aazuspan aazuspan self-assigned this Sep 20, 2021
@aazuspan
Copy link
Owner Author

aazuspan commented Oct 6, 2021

This is more complicated because the ee.batch.Export.image.toDrive method has no filePerBand option, unlike getDownloadURL, and the _dataset_from_files function requires one file per band. The _dataset_from_files function will have to be adjusted to allow multiband files.

At that point, I need to decide whether to adjust TimeSeries.to_xarray to run with multiband files for consistency or leave as is with some methods using multiband files and others using file-per-band...

aazuspan added a commit that referenced this issue Oct 8, 2021
@ddjustina

This comment was marked as off-topic.

@aazuspan

This comment was marked as off-topic.

@aazuspan aazuspan removed their assignment Mar 8, 2023
@aazuspan
Copy link
Owner Author

aazuspan commented Mar 8, 2023

I'm closing this as out of scope. There are existing tools for exporting image collections to drive and loading a set of images into an xarray.Dataset is pretty straightforward, so I don't think wxee adds enough utility here to be worth pursuing.

@aazuspan aazuspan closed this as not planned Won't fix, can't repro, duplicate, stale Mar 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants