req_throttle does not throttle requests even when no tokens are available

[mattwarkentin]

Hi,

I am not sure if I am doing something wrong or if I just don't understand how req_throttle() works. But my expectation was that there would only be 1 request per minute based on this configuration. But what happens is that requests are made as fast as possible until the rate limit is hit, even when no tokens are available via throttle_status(). Is this a bug or am I doing something wrong?

library(httr2)

foo <- function() {
  x <-
    request("https://eutils.ncbi.nlm.nih.gov/entrez/eutils") |>
    req_url_path_append("esearch.fcgi") |>
    req_url_query(
      db = 'pubmed',
      term = 'Richard Feynman[au]',
      retmode = 'json'
    ) |>
    req_throttle(capacity = 1, fill_time_s = 60) |>
    req_perform() |>
    resp_body_json()

  print(throttle_status())

  x
}

replicate(100, foo())
#>                     realm tokens to_wait
#> 1 eutils.ncbi.nlm.nih.gov      0      60
#>                     realm tokens to_wait
#> 1 eutils.ncbi.nlm.nih.gov      0      59
#>                     realm tokens to_wait
#> 1 eutils.ncbi.nlm.nih.gov      0      60
#> Error in `req_perform()`:
#> ! HTTP 429 Too Many Requests.

[hadley]

Hmmmm, looks like there's a little bug somewhere, because this works as expected:

library(httr2)

req <- request("https://eutils.ncbi.nlm.nih.gov/entrez/eutils") |>
  req_url_path_append("esearch.fcgi") |>
  req_url_query(
    db = 'pubmed',
    term = 'Richard Feynman[au]',
    retmode = 'json'
  ) |>
  req_throttle(capacity = 1, fill_time_s = 60)

foo <- function() {
  req |> req_perform() |> resp_body_json()
  print(throttle_status())
}
foo()
foo()

Looks like I'm accidentally resetting the bucket every time req_throttle() is called:

the$throttle[[realm]] <- TokenBucket$new(capacity, fill_time_s)
# probably should  be:
the$throttle[[realm]] <- the$throttle[[realm]] %||% TokenBucket$new(capacity, fill_time_s)

But I need to think through what happens if the capacity or fill time has changed.

In any case, in the mean time you can work around the problem by creating a single request and reusing it.

[mattwarkentin]

Right, I guess every time req_throttle() is invoked when the wrapping function is called it resets the bucket and the requests can happen immediately. Building the request outside of the function call navigates around that.

This toy example is a simplification of what I'm hoping to do - wrap some API endpoints into functions in an R package. I wanted the package/function to handle the throttling.

Is this a breaking change from how throttling was handled previously by {httr2}? Here is the example provided in the "Wrapping APIs" article:

nytimes_books <- function(api_key, path, ...) {
  request("https://api.nytimes.com/svc/books/v3") |>
    req_url_path_append(path) |>
    req_url_query(..., `api-key` = api_key) |>
    req_error(body = nytimes_error_body) |>
    req_throttle(10 / 60, realm = "https://api.nytimes.com/svc/books") |>
    req_perform() |>
    resp_body_json()
}

The throttling is handled inside the function yet I presume repeated function calls still adhered to rate limits.

[hadley]

Yes, as I said, it's a bug.

[mattwarkentin]

Okay, sorry, I was just trying to clarify for my own understanding.

[giannetti]

I experienced this issue too. I got around it using Sys.sleep() inside a for loop.