Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update check_images_available to be able check for images available for entire date range #490

Merged
merged 1 commit into from
Apr 18, 2024

Conversation

2320sharon
Copy link
Contributor

Hi Killian!

Thanks for building out the functionality to check for existing imagery and continue downloads from the point the download was interrupted. I finally got this feature integrated in coastsat-package and coastseg.

When I was updating coastsat-package's code to allow for downloads to resume I realized that code in check_images_available that checks for existing imagery doesn't actually check if the entire requested date range has been downloaded due to this line

idx_new = np.where([np.logical_or(_< first_date, _ > last_date) for _ in date_list])[0]

Basically the imagery doesn't always download chronically meaning that it is possible that first date or last date that was downloaded isn't the actual first and last date of the requested date range. If this happens the partial download won't download the missing images for the entire requested date range instead it will only download the missing images for the date range that had already been downloaded for that satellite. You can see that first and last date are derived from the existing imagery in the code below from check_images_available:

first_date = metadata_existing[satname]['dates'][0] - timedelta(days=1)
last_date = metadata_existing[satname]['dates'][-1] + timedelta(days=1)

What this new code does is it gets the entire available date range then gets the existing downloaded dates from the imagery and then it filters out the dates that have already been downloaded. This has been working great for us over at CoastSeg and always gets the entire date range available.

I hope this helps!

Thanks for maintaining CoastSat for all these years!!

@kvos
Copy link
Owner

kvos commented Apr 18, 2024

hi Sharon! great that you picked that up, thanks for the PR!!!

@kvos kvos merged commit 84972fa into kvos:master Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants