Skip to content

/ hereR

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adjust .get_content() to handle failing requests #17

Closed
munterfinger opened this issue Dec 8, 2019 · 2 comments
Closed

Adjust .get_content() to handle failing requests #17

munterfinger opened this issue Dec 8, 2019 · 2 comments
Assignees
@munterfinger
Labels
bug

Comments

@munterfinger
Copy link
Owner

@munterfinger munterfinger commented Dec 8, 2019

If a requests fails, then this entry is deleted from the API result list. This means that the order of the output is not correct, which disables joining the output to the input based on the "id" column.

The following code has to be adjusted:

.get_content <- function(url, encoding = "UTF-8") {
...
      if (res$status != 200) {
        warning(sprintf("Request failed: HTTP status code %s.", res$status))
        ids <<- ids[ids != id]
...
}

The output of .get_content() has to be of the same length as the input URLs. The list entry of a failing request has to exist, but can be empty.
@munterfinger munterfinger self-assigned this Dec 8, 2019
@munterfinger munterfinger added the bug label Dec 8, 2019
@munterfinger munterfinger mentioned this issue Dec 8, 2019
Closed
@munterfinger
Copy link
Owner Author

@munterfinger munterfinger commented Dec 15, 2019 •
edited

Construct a dummy data set with points in the ocean:

x <- data.frame(
  id = c(1,2,3,4,5,6),
  place = c("Barcelona1", "Barcelona2", "Ocean1", "Ocean2", "Melbourne1", "Melbourne2"),
  lng = c(2.1, 2.173404, 76, 77, 144.963058, 145),
  lat = c(41.4, 41.385063, -28, -27, -37.813629, -37.5)
, stringsAsFactors = FALSE) %>% st_as_sf(coords = c("lng", "lat")) %>% st_set_crs(4326)

Checking geocode():

y <- geocode(x$place)
  • Change id to ids[count]

Checking geocode():

y <- autocomplete(x$place)
  • Add id

Checkingreverse_geocode():

y <- reverse_geocode(x, landmarks = FALSE)
y <- reverse_geocode(x, landmarks = TRUE)
  • Change id to ids[count]

Checking route():

y <- route(x[c(1, 3, 5), ], x[c(2, 4, 6), ])
#> Warning in (function (res) : Request failed: HTTP status code 400.
  • Add id

Checking route_matrix():

y <- route_matrix(x)
  • Nothing to do

Checking isoline():

y <- isoline(x, aggregate = FALSE)
#> Warning in (function (res) : Request failed: HTTP status code 400.
#> Warning in (function (res) : Request failed: HTTP status code 400.
  • Add id

Checking traffic():

y <- st_buffer(x, 0.01)
#> Warning in st_buffer.sfc(st_geometry(x), dist, nQuadSegs, endCapStyle =
#> endCapStyle, : st_buffer does not correctly buffer longitude/latitude data
#> dist is assumed to be in decimal degrees (arc_degrees).
z <- traffic(y, product = "flow")
#> Warning in (function (res) : Request failed: HTTP status code 204.
#> Warning in (function (res) : Request failed: HTTP status code 204.
#> Warning in (function (res) : Request failed: HTTP status code 204.
z <- traffic(y, product = "incidents")
  • Add id

Checking weather():

y <- weather(x, product = "observation")
y <- weather(x, product = "forecast_hourly")
y <- weather(x, product = "forecast_astronomy")
y <- weather(x, product = "alerts")
  • Add id
Session info 
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.1 (2019-07-05)
#>  os       macOS Catalina 10.15.1      
#>  system   x86_64, darwin15.6.0        
#>  ui       X11                         
#>  language (EN)                        
#>  collate  de_CH.UTF-8                 
#>  ctype    de_CH.UTF-8                 
#>  tz       Europe/Zurich               
#>  date     2019-12-15                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version    date       lib source        
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 3.6.0)
#>  backports     1.1.5      2019-10-02 [1] CRAN (R 3.6.0)
#>  callr         3.3.2      2019-09-22 [1] CRAN (R 3.6.0)
#>  class         7.3-15     2019-01-01 [1] CRAN (R 3.6.1)
#>  classInt      0.4-2      2019-10-17 [1] CRAN (R 3.6.0)
#>  cli           1.1.0      2019-03-19 [1] CRAN (R 3.6.0)
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 3.6.0)
#>  curl          4.2        2019-09-24 [1] CRAN (R 3.6.0)
#>  data.table    1.12.6     2019-10-18 [1] CRAN (R 3.6.0)
#>  DBI           1.0.0      2018-05-02 [1] CRAN (R 3.6.0)
#>  desc          1.2.0      2018-05-01 [1] CRAN (R 3.6.0)
#>  devtools      2.2.1      2019-09-24 [1] CRAN (R 3.6.0)
#>  digest        0.6.22     2019-10-21 [1] CRAN (R 3.6.0)
#>  e1071         1.7-2      2019-06-05 [1] CRAN (R 3.6.0)
#>  ellipsis      0.3.0      2019-09-20 [1] CRAN (R 3.6.0)
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 3.6.0)
#>  fs            1.3.1      2019-05-06 [1] CRAN (R 3.6.0)
#>  glue          1.3.1      2019-03-12 [1] CRAN (R 3.6.0)
#>  hereR       * 0.2.0.9000 2019-12-11 [1] local         
#>  htmltools     0.4.0      2019-10-04 [1] CRAN (R 3.6.0)
#>  jsonlite      1.6        2018-12-07 [1] CRAN (R 3.6.0)
#>  KernSmooth    2.23-16    2019-10-15 [1] CRAN (R 3.6.0)
#>  knitr         1.25       2019-09-18 [1] CRAN (R 3.6.0)
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 3.6.0)
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 3.6.0)
#>  pkgbuild      1.0.6      2019-10-09 [1] CRAN (R 3.6.0)
#>  pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.6.0)
#>  prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.6.0)
#>  processx      3.4.1      2019-07-18 [1] CRAN (R 3.6.0)
#>  ps            1.3.0      2018-12-21 [1] CRAN (R 3.6.0)
#>  R6            2.4.1      2019-11-12 [1] CRAN (R 3.6.0)
#>  Rcpp          1.0.3      2019-11-08 [1] CRAN (R 3.6.0)
#>  remotes       2.1.0      2019-06-24 [1] CRAN (R 3.6.0)
#>  rlang         0.4.1      2019-10-24 [1] CRAN (R 3.6.0)
#>  rmarkdown     1.16       2019-10-01 [1] CRAN (R 3.6.0)
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.6.0)
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.6.0)
#>  sf          * 0.8-0      2019-09-17 [1] CRAN (R 3.6.0)
#>  stringi       1.4.3      2019-03-12 [1] CRAN (R 3.6.0)
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 3.6.0)
#>  testthat      2.2.1      2019-07-25 [1] CRAN (R 3.6.0)
#>  units         0.6-5      2019-10-08 [1] CRAN (R 3.6.0)
#>  usethis       1.5.1      2019-07-04 [1] CRAN (R 3.6.0)
#>  withr         2.1.2      2018-03-15 [1] CRAN (R 3.6.0)
#>  xfun          0.10       2019-10-01 [1] CRAN (R 3.6.0)
#>  yaml          2.2.0      2018-07-25 [1] CRAN (R 3.6.0)
#> 
#> [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
@munterfinger
Copy link
Owner Author

@munterfinger munterfinger commented Dec 15, 2019

Fixed the handling of failing requests in .get_content(). The "id"
column is still in correct order, even if there are failing requests in
a function call.

Construct a dummy data set with points in the ocean:

x <- data.frame(
  id = c(1,2,3,4,5,6),
  address = c("Barcelona", "Barcelona", "not_an_address1", "not_an_address2", "Melbourne", "Melbourne"),
  place = c("Barcelona1", "Barcelona2", "Ocean1", "Ocean2", "Melbourne1", "Melbourne2"),
  lng = c(2.12, 2.173404, 76, 77, 144.963058, 145),
  lat = c(41.4, 41.385063, -28, -27, -37.813629, -37.5)
, stringsAsFactors = FALSE) %>% st_as_sf(coords = c("lng", "lat")) %>% st_set_crs(4326)

Define a function which prints the ids of the input and output:

print_ids <- function(obj) {
  message("*** Compare ids ***")
  message("Input:  ", paste(unique(x$id), collapse = ", "))
  message("Output: ", paste(unique(obj$id), collapse = ", "))
}

Checking geocode():

print_ids(y <- geocode(x$address))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Address(es) 'not_an_address1', 'not_an_address2' not found.
#> Output: 1, 2, 5, 6

Checkingreverse_geocode():

print_ids(y <- reverse_geocode(x, landmarks = FALSE))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Output: 1, 2, 5, 6
print_ids(y <- reverse_geocode(x, landmarks = TRUE))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Output: 1, 2, 5, 6

Checking route():

print_ids(y <- route(x[c(1, 3, 5), ], x[c(2, 4, 6), ]))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Warning in (function (res) : Request 'id = 2' failed: Status 400.
#> Output: 1, 3

Checking isoline():

print_ids(y <- isoline(x, aggregate = TRUE))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Warning in (function (res) : Request 'id = 4' failed: Status 400.
#> Warning in (function (res) : Request 'id = 3' failed: Status 400.
#> Output: NA
print_ids(y <- isoline(x, aggregate = FALSE))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Warning in (function (res) : Request 'id = 4' failed: Status 400.
#> Warning in (function (res) : Request 'id = 3' failed: Status 400.
#> Output: 1, 2, 5, 6

Checking traffic():

y <- st_buffer(x, 0.01)
#> Warning in st_buffer.sfc(st_geometry(x), dist, nQuadSegs, endCapStyle =
#> endCapStyle, : st_buffer does not correctly buffer longitude/latitude data
#> dist is assumed to be in decimal degrees (arc_degrees).
print_ids(z <- traffic(y, product = "flow"))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Warning in (function (res) : Request 'id = 6' failed: Status 204.
#> Warning in (function (res) : Request 'id = 3' failed: Status 204.
#> Warning in (function (res) : Request 'id = 4' failed: Status 204.
#> Output: 1, 2, 5
print_ids(z <- traffic(y, product = "incidents"))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Output: 2, 5

Checking weather():

print_ids(y <- weather(x, product = "observation"))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Output: 1, 2, 3, 4, 5, 6
print_ids(y <- weather(x, product = "forecast_hourly"))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Output: 1, 2, 3, 4, 5, 6
print_ids(y <- weather(x, product = "forecast_astronomy"))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Output: 1, 2, 3, 4, 5, 6
print_ids(y <- weather(x, product = "alerts"))
#> *** Compare ids ***
#> Input:  1, 2, 3, 4, 5, 6
#> Output: 1, 2, 3, 4, 5, 6
Session info 
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.1 (2019-07-05)
#>  os       macOS Catalina 10.15.1      
#>  system   x86_64, darwin15.6.0        
#>  ui       X11                         
#>  language (EN)                        
#>  collate  de_CH.UTF-8                 
#>  ctype    de_CH.UTF-8                 
#>  tz       Europe/Zurich               
#>  date     2019-12-15                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version    date       lib source        
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 3.6.0)
#>  backports     1.1.5      2019-10-02 [1] CRAN (R 3.6.0)
#>  callr         3.3.2      2019-09-22 [1] CRAN (R 3.6.0)
#>  class         7.3-15     2019-01-01 [1] CRAN (R 3.6.1)
#>  classInt      0.4-2      2019-10-17 [1] CRAN (R 3.6.0)
#>  cli           1.1.0      2019-03-19 [1] CRAN (R 3.6.0)
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 3.6.0)
#>  curl          4.2        2019-09-24 [1] CRAN (R 3.6.0)
#>  data.table    1.12.6     2019-10-18 [1] CRAN (R 3.6.0)
#>  DBI           1.0.0      2018-05-02 [1] CRAN (R 3.6.0)
#>  desc          1.2.0      2018-05-01 [1] CRAN (R 3.6.0)
#>  devtools      2.2.1      2019-09-24 [1] CRAN (R 3.6.0)
#>  digest        0.6.22     2019-10-21 [1] CRAN (R 3.6.0)
#>  e1071         1.7-2      2019-06-05 [1] CRAN (R 3.6.0)
#>  ellipsis      0.3.0      2019-09-20 [1] CRAN (R 3.6.0)
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 3.6.0)
#>  fs            1.3.1      2019-05-06 [1] CRAN (R 3.6.0)
#>  glue          1.3.1      2019-03-12 [1] CRAN (R 3.6.0)
#>  hereR       * 0.2.0.9000 2019-12-15 [1] local         
#>  htmltools     0.4.0      2019-10-04 [1] CRAN (R 3.6.0)
#>  jsonlite      1.6        2018-12-07 [1] CRAN (R 3.6.0)
#>  KernSmooth    2.23-16    2019-10-15 [1] CRAN (R 3.6.0)
#>  knitr         1.25       2019-09-18 [1] CRAN (R 3.6.0)
#>  lwgeom        0.1-7      2019-05-06 [1] CRAN (R 3.6.0)
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 3.6.0)
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 3.6.0)
#>  pkgbuild      1.0.6      2019-10-09 [1] CRAN (R 3.6.0)
#>  pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.6.0)
#>  prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.6.0)
#>  processx      3.4.1      2019-07-18 [1] CRAN (R 3.6.0)
#>  ps            1.3.0      2018-12-21 [1] CRAN (R 3.6.0)
#>  R6            2.4.1      2019-11-12 [1] CRAN (R 3.6.0)
#>  Rcpp          1.0.3      2019-11-08 [1] CRAN (R 3.6.0)
#>  remotes       2.1.0      2019-06-24 [1] CRAN (R 3.6.0)
#>  rlang         0.4.1      2019-10-24 [1] CRAN (R 3.6.0)
#>  rmarkdown     1.16       2019-10-01 [1] CRAN (R 3.6.0)
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.6.0)
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.6.0)
#>  sf          * 0.8-0      2019-09-17 [1] CRAN (R 3.6.0)
#>  stringi       1.4.3      2019-03-12 [1] CRAN (R 3.6.0)
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 3.6.0)
#>  testthat      2.2.1      2019-07-25 [1] CRAN (R 3.6.0)
#>  units         0.6-5      2019-10-08 [1] CRAN (R 3.6.0)
#>  usethis       1.5.1      2019-07-04 [1] CRAN (R 3.6.0)
#>  withr         2.1.2      2018-03-15 [1] CRAN (R 3.6.0)
#>  xfun          0.10       2019-10-01 [1] CRAN (R 3.6.0)
#>  yaml          2.2.0      2018-07-25 [1] CRAN (R 3.6.0)
#> 
#> [1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@munterfinger munterfinger

Labels
bug
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
@munterfinger
You can’t perform that action at this time.