req_throttle() error: take_token() sometimes returns negative values

[nielsoledam]

Reprex:

library(httr2)
# Setup the query
url <- "https://whatever.com"
req <- request(url) |>
    req_throttle(capacity = 5, fill_time_s = 5)

Sys.sleep(2)

# Simulate fast use of req_throttle tokens
for (n in 1:15) {
    print(httr2:::the$throttle[[req$policies$throttle_realm]]$take_token())
}
#> [1] 0
#> [1] 0
#> [1] 0
#> [1] 0
#> [1] 0
#> [1] -1
#> [1] 0
#> [1] 1
#> [1] 2
#> [1] 3
#> [1] 4
#> [1] 5
#> [1] 6
#> [1] 7
#> [1] 8

^{Created on 2025-12-05 with reprex v2.1.1}

Why this is an issue

This is an issue because the internal httr2:::sys_sleep() function used internally by req_perform() does not handle that edge case gracefully:

...
sys_sleep(throttle_delay(req), "for throttling delay")
...

(see: L104)

...as it only checks explicitly for time == 0 and therefore just goes ahead and feeds negative values into Sys.sleep() – as shown below:

httr2:::sys_sleep
#> function (seconds, task, fps = 10, progress = NULL) 
#> {
#>     check_number_decimal(seconds)
#>     check_string(task)
#>     check_number_decimal(fps)
#>     progress <- progress %||% getOption("httr2_progress", !is_testing())
#>     check_bool(progress, allow_null = TRUE)
#>     if (seconds == 0) {
#>         return(invisible())
#>     }
#>     if (!progress) {
#>         cli::cli_alert("Waiting {round(seconds, 2)}s {task}")
#>         Sys.sleep(seconds)
#>         return(invisible())
#>     }
#>     start <- cur_time()
#>     signal("", class = "httr2_sleep", seconds = seconds)
#>     cli::cli_progress_bar(format = "Waiting {ceiling(seconds)}s {task} {cli::pb_bar}", 
#>         total = seconds * fps)
#>     while ({
#>         left <- start + seconds - cur_time()
#>         left > 0
#>     }) {
#>         Sys.sleep(min(1/fps, left))
#>         cli::cli_progress_update(set = (seconds - left) * fps)
#>     }
#>     cli::cli_progress_done()
#>     invisible()
#> }
#> <bytecode: 0x110a958a0>
#> <environment: namespace:httr2>

^{Created on 2025-12-05 with reprex v2.1.1}

How I found the issue

My self-hosted github runner regularly throws this error:

Workarounds

I haven't found any yet.

(Except to stop using req_throttle, of course.)

[hadley]

Here's some code you can run inside of an httr2 dev environment to reliably illustrate the problem:

mock_time <- 0
local_mocked_bindings(unix_time = function() mock_time)
throttle_reset("x.com")

req <- request("https://x.com") |> req_throttle(capacity = 5, fill_time_s = 5)
mock_time <- 3

for (n in 1:10) {
  print(throttle_delay(req))
}

It looks like it's the first token after a reset that's the problem, and it seems to be relating to waiting some time in between setting up the throttle and requesting a delay. I suspect I'm missing a max(something, 0) somewhere.

[nielsoledam]

Wow, that was fast!
Solved the problem – no errors on my github runner during the latest run.
Thanks for swift action!
:-)