Error "Index out of bounds" when applying a property filter

[hewag1975]

Hi Marius,

if the result of applying a property filter on gdalcubes::stac_image_collection is empty, a cryptic error message from the underlying C-function occurs. See the reprex below.

library(rstac)
library(gdalcubes)

# AOI bbox
bbox = c(
    "xmin" = 11.14079
    , "ymin" = 49.75891
    , "xmax" = 11.14321
    , "ymax" = 49.76039
)

cloudcover = 20

# STAC
s = stac("https://earth-search.aws.element84.com/v0")

# example with Sentinel Collection and period without images that have cloudcover <= 20%
items_sen = s |>
    stac_search(collections = "sentinel-s2-l2a-cogs",
                bbox = bbox,
                datetime = "2021-10-15/2021-10-30",
                limit = 500) |>
    post_request()

collection = stac_image_collection(
    items_sen$features, 
    property_filter = function(x) {x[["eo:cloud_cover"]] <= cloudcover}
)
#> Error in libgdalcubes_create_stac_collection(bands_df, images_df, gdalrefs_df, : Index out of bounds: [index='id'].

^{Created on 2021-11-03 by the reprex package (v2.0.1)}

Session info

sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.1.2 (2021-11-01)
#>  os       Ubuntu 20.04.3 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language en_US:en                    
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       Europe/Berlin               
#>  date     2021-11-03                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package      * version date       lib source        
#>  backports      1.2.1   2020-12-09 [2] CRAN (R 4.1.1)
#>  cli            3.0.1   2021-07-17 [2] CRAN (R 4.1.1)
#>  crayon         1.4.1   2021-02-08 [2] CRAN (R 4.1.1)
#>  curl           4.3.2   2021-06-23 [1] CRAN (R 4.1.1)
#>  digest         0.6.28  2021-09-23 [1] CRAN (R 4.1.1)
#>  ellipsis       0.3.2   2021-04-29 [2] CRAN (R 4.1.1)
#>  evaluate       0.14    2019-05-28 [2] CRAN (R 4.1.1)
#>  fansi          0.5.0   2021-05-25 [2] CRAN (R 4.1.1)
#>  fastmap        1.1.0   2021-01-25 [2] CRAN (R 4.1.1)
#>  fs             1.5.0   2020-07-31 [1] CRAN (R 4.1.1)
#>  gdalcubes    * 0.5.0   2021-10-28 [1] CRAN (R 4.1.1)
#>  glue           1.4.2   2020-08-27 [1] CRAN (R 4.1.1)
#>  highr          0.9     2021-04-16 [2] CRAN (R 4.1.1)
#>  htmltools      0.5.2   2021-08-25 [2] CRAN (R 4.1.1)
#>  httr           1.4.2   2020-07-20 [1] CRAN (R 4.1.1)
#>  jsonlite       1.7.2   2020-12-09 [1] CRAN (R 4.1.1)
#>  knitr          1.36    2021-09-29 [1] CRAN (R 4.1.1)
#>  lifecycle      1.0.1   2021-09-24 [1] CRAN (R 4.1.1)
#>  magrittr       2.0.1   2020-11-17 [1] CRAN (R 4.1.1)
#>  ncdf4          1.17    2019-10-23 [1] CRAN (R 4.1.1)
#>  pillar         1.6.2   2021-07-29 [2] CRAN (R 4.1.1)
#>  pkgconfig      2.0.3   2019-09-22 [2] CRAN (R 4.1.1)
#>  purrr          0.3.4   2020-04-17 [1] CRAN (R 4.1.1)
#>  R.cache        0.15.0  2021-04-30 [1] CRAN (R 4.1.1)
#>  R.methodsS3    1.8.1   2020-08-26 [2] CRAN (R 4.1.1)
#>  R.oo           1.24.0  2020-08-26 [2] CRAN (R 4.1.1)
#>  R.utils        2.11.0  2021-09-26 [1] CRAN (R 4.1.1)
#>  R6             2.5.1   2021-08-19 [1] CRAN (R 4.1.1)
#>  Rcpp           1.0.7   2021-07-07 [1] CRAN (R 4.1.1)
#>  RcppProgress   0.4.2   2020-02-06 [1] CRAN (R 4.1.1)
#>  reprex         2.0.1   2021-08-05 [1] CRAN (R 4.1.1)
#>  rlang          0.4.12  2021-10-18 [1] CRAN (R 4.1.1)
#>  rmarkdown      2.11    2021-09-14 [1] CRAN (R 4.1.1)
#>  rstac        * 0.9.1-5 2021-10-31 [1] CRAN (R 4.1.1)
#>  rstudioapi     0.13    2020-11-12 [2] CRAN (R 4.1.1)
#>  sessioninfo    1.1.1   2018-11-05 [2] CRAN (R 4.1.1)
#>  stringi        1.7.5   2021-10-04 [1] CRAN (R 4.1.1)
#>  stringr        1.4.0   2019-02-10 [1] CRAN (R 4.1.1)
#>  styler         1.6.2   2021-09-23 [1] CRAN (R 4.1.1)
#>  tibble         3.1.5   2021-09-30 [1] CRAN (R 4.1.1)
#>  utf8           1.2.2   2021-07-24 [1] CRAN (R 4.1.1)
#>  vctrs          0.3.8   2021-04-29 [2] CRAN (R 4.1.1)
#>  withr          2.4.2   2021-04-18 [1] CRAN (R 4.1.1)
#>  xfun           0.26    2021-09-14 [2] CRAN (R 4.1.1)
#>  yaml           2.2.1   2020-02-01 [1] CRAN (R 4.1.1)
#> 
#> [1] /home/hendrik/R/x86_64-pc-linux-gnu-library/4.1
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library

By playing around with the threshold for cloud cover we observed that there are simply no collections available within the selected period / AOI and below the cloud cover threshold. We were able to deal with it internally, still I think the message should be somewhat informative to the user.

Thanks,
Hendrik

[appelmar]

Thanks, this is indeed confusing to users. As you suggest, I'll add an informative error message and simply stop the function if there are no items. The alternative to silently return an empty image collection would probably also lead to confusion later when resulting cubes are NA only...

[hewag1975]

Thanks! Looks good to me.