Customizable timeout for pkg_search/advanced_search

[gladkia]

Overview

HI @gaborcsardi. Thanks for creating such a great package. I have a suggestion for a small improvement.

The R_PKG_SEARCH_SERVER can occasionally be unavailable link.

The curl::curl_fetch_memory has a long timeout set by default (10[s]).

> curl::curl_fetch_memory(test_url)
Error in curl::curl_fetch_memory(test_url) : 
  Timeout was reached: [gladki.pl:4321] Connection timeout after 10001 ms

Fortunately, the timeout setting can be easily changed

> new_handle <- curl::new_handle(url = test_url)
> curl::handle_setopt(
+   new_handle,
+   .list = list(timeout = 2)
+ )
> curl::curl_fetch_memory(test_url, handle = new_handle)
Error in curl::curl_fetch_memory(test_url, handle = new_handle) : 
  Timeout was reached: [gladki.pl:4321] Connection timed out after 2002 milliseconds

Proposed solution

Add the possibility to change the timeout for the curl's connect phase. With such a change, if R_PKG_SEARCH_SERVER becomes unavailable, one can spot the error faster than within 10[s].

[gaborcsardi]

I don't anticipate it to go offline again, now that it has found a good home.

Nevertheless, it is already possible to change the timeout with the timeout option, so I am not sure if we need to add another option that is specific to pkgsearch.

[gladkia]

Great news!

How can I change the timeout? I've tried with the timeout param (the latest main). No success, unfortunately.
Details:

> setwd("~/projects/pkgsearch/")
> pkgload::load_all("~/projects/pkgsearch/")
ℹ Loading pkgsearch
> git2r::last_commit()
[593cb84] 2024-04-20: Use HTTPS for the search server
> pkgsearch::advanced_search(Author = "John", Author = "Doe")
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Timeout was reached: [search.r-pkg.org:4321] Connection timeout after 10001 ms
> pkgsearch::advanced_search(Author = "John", Author = "Doe", timeout = 2)
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Timeout was reached: [search.r-pkg.org:4321] Connection timeout after 10002 ms
> Sys.getenv("R_PKG_SEARCH_SERVER")
[1] "search.r-pkg.org:4321"
> Sys.unsetenv("R_PKG_SEARCH_SERVER")
> pkgsearch::advanced_search(Author = "John", Author = "Doe", timeout = 2)
- "advanced search" ---------------------------------------------------------------------- 0 packages in 0.002 seconds -

[gaborcsardi]

It is a global option:

options(timeout = 2)

See ?options.

[gladkia]

It does not work for me.

> options(timeout = 2)
> getOption("timeout")
[1] 2
> Sys.setenv(R_PKG_SEARCH_SERVER = "search.r-pkg.org:4321")
> pkgsearch::advanced_search(Author = "John", Author = "Doe")
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Timeout was reached: [search.r-pkg.org:4321] Failed to connect to search.r-pkg.org port 4321 after 10001 ms: Timeout was reached
> Sys.getenv("R_PKG_SEARCH_SERVER")
[1] "search.r-pkg.org:4321"
> Sys.unsetenv("R_PKG_SEARCH_SERVER")
> pkgsearch::advanced_search(Author = "John", Author = "Doe")
- "advanced search" ---------------------------------------------------------------------- 0 packages in 0.072 seconds -

[gaborcsardi]

Ohh, you are right, I mixed it up with HTTPUserAgent, which curl respects. But it does not respect timeout.

I think we can respect timeout in pkgsearch, the default one minute should be plenty, even if your internet connection is slow.

[gladkia]

@gaborcsardi - I've updated the PR accordingly: https://github.com/r-hub/pkgsearch/pull/125/files.