Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First stab at req_perform_sequential() #361

Merged
merged 12 commits into from
Oct 30, 2023
4 changes: 3 additions & 1 deletion R/multi-req.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@
#' * Consults the cache set by [req_cache()] before/after all requests.
#'
#' In general, where [req_perform()] might make multiple requests due to retries
#' or OAuth failures, `req_perform_parallel()` will only make 1.
#' or OAuth failures, `req_perform_parallel()` will only make 1. If any of
#' these limitations are problematic, you may want to use
#' [req_perform_sequential()] instead.
#'
#' @param reqs A list of [request]s.
#' @param paths An optional list of paths, if you want to download the request
Expand Down
48 changes: 48 additions & 0 deletions R/sequential.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#' Perform multiple requests in sequence
#'
#' Given a list of requests, this function requests each one turn, returning
#' a list of responses.
#'
#' @inheritParams req_perform_sequential
#' @inheritParams req_perform_iteratively
#' @export
#' @examples
#' req <- request("https://api.restful-api.dev/objects")
#' ids <- sort(sample(100, 50))
#' chunks <- split(ids, (seq_along(ids) - 1) %/% (length(ids) / 10))
#' reqs <- chunks %>% lapply(\(idx) req %>% req_url_query(id = idx, .multi = "comma"))
#' resps <- reqs %>% req_perform_sequential()
#' resps_data(resps, \(resp) resp_body_json(resp))
req_perform_sequential <- function(reqs, paths = NULL, progress = TRUE) {
if (!is.list(reqs)) {
stop_input_type(reqs, "a list")
}
if (!is.null(paths)) {
if (length(reqs) != length(paths)) {
hadley marked this conversation as resolved.
Show resolved Hide resolved
cli::cli_abort("If supplied, {.arg paths} must be the same length as {.arg req}.")
}
}

progress <- create_progress_bar(
total = length(reqs),
name = "Iterating",
config = progress
)

resps <- rep_along(reqs, list())

tryCatch({
for (i in seq_along(reqs)) {
check_request(reqs[[i]], arg = glue::glue("req[[{i}]]"))
resps[[i]] <- req_perform(reqs[[i]], path = paths[[i]])
progress$update()
}
}, interrupt = function(cnd) {
cli::cli_alert_warning(
hadley marked this conversation as resolved.
Show resolved Hide resolved
"Terminating iteration; returning {i} response{?s}."
)
})
progress$done()

resps
}
Loading