Avoid downloading files that exist with byTileAOP?

[mbjoseph]

Is your feature request related to a problem? Please describe.

Currently byTileAOP, downloads files that exist already. It might be nice to add an overwrite argument that defaults to the current behavior (overwrite = TRUE), but allows users to skip downloads for files that already exist using overwrite = FALSE.

Describe the solution you'd like

I'd propose adding an overwrite argument to byTileAOP.

Describe alternatives you've considered

Currently, it's up to the users to avoid redundant downloads. My use case is trying to get canopy height model rasters at locations of ground plots. But, in some cases, multiple plots fall within the same tile, resulting in redundant downloads. As a user, I could do something like:

1. Start by downloading one tile for the first ground plot location
2. Read the polygon of the CHM from the metadata and remove any plots that are within the boundary of the CHM I just downloaded
3. Iterate to the next ground plot and go back to step 1

Additional context

Here's an example of what I'm doing -- essentially I'm computing one coordinate for each ground plot, then I want to get the CHM data for each one, which results in redundant downloads:

library(tidyverse)
library(neonUtilities)
library(geoNEON)

# Get NEON CHM data for SJER, over the ground plots
# (adapted from https://www.neonscience.org/tree-heights-veg-structure-chm)
veglist <- loadByProduct(dpID="DP1.10098.001", site="SJER", package="basic")
vegmap <- def.calc.geo.os(veglist$vst_mappingandtagging, 
                          "vst_mappingandtagging")

# Compute one coordinate per plot
plot_coords <- vegmap %>%
  group_by(plotID) %>%
  summarize(easting = mean(adjEasting, na.rm = TRUE), 
            northing = mean(adjNorthing, na.rm = TRUE))

# For each plot, download the CHM data (results in redundant downloads)
for (i in 1:nrow(plot_coords)) {
  byTileAOP(dpID="DP3.30015.001", 
            site="SJER", 
            year="2017", 
            easting=plot_coords$easting[i], 
            northing=plot_coords$northing[i],
            savepath="data", 
            check.size = FALSE)
}

Maybe there's a better way to handle the possibility of redundant downloads in this case, but this seems like a potentially general solution for similar use cases.

[cklunch]

@mbjoseph Looking at your code, if I'm understanding right, I think there's an easy solution, and I just need to improve the documentation. byTileAOP can take vector inputs for easting and northing. It will assume the vectors line up (ie, easting[1] and northing[1] are a desired pair of coordinates), but that should be accurate. So you can just skip the for loop, and the function will come up with the set of tiles that contain all the input coordinates.

But let me know if I've misunderstood!

[mbjoseph]

Oh -- nice! Nevermind then. I didn't realize you could pass a vector of coordinates.

…

On Sun, Oct 20, 2019, 11:56 AM Claire Lunch ***@***.***> wrote: @mbjoseph <https://github.com/mbjoseph> Looking at your code, if I'm understanding right, I think there's an easy solution, and I just need to improve the documentation. byTileAOP can take vector inputs for easting and northing. It will assume the vectors line up (ie, easting[1] and northing[1] are a desired pair of coordinates), but that should be accurate. So you can just skip the for loop, and the function will come up with the set of tiles that contain all the input coordinates. But let me know if I've misunderstood! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#61?email_source=notifications&email_token=AAUKQ5GGRAWXE7LEDOJYUKDQPSSWTA5CNFSM4JCVOWT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBYRDWQ#issuecomment-544281050>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUKQ5ESQ54FH4ECFUJHPK3QPSSWTANCNFSM4JCVOWTQ> .

[cklunch]

No problem, glad to hear this solves this issue!