New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: easy handled already used in multi handle #112

Closed
noamross opened this Issue Jul 4, 2014 · 16 comments

Comments

Projects
None yet
@noamross

noamross commented Jul 4, 2014

Sometimes GET() returns the error:

Error in function (type, msg, asError = TRUE)  : 
  easy handled already used in multi handle

I'm having trouble isolating the exact cause, but it seems to occur when I've performed at least one GET() request in my session, possibly only if I've aborted the function before it completed.

@hadley

This comment has been minimized.

Member

hadley commented Jul 4, 2014

The error appears to be coming from this line in libcurl: https://github.com/bagder/curl/blob/master/lib/easy.c#L762. So something must be incorrectly setting multi to true (or perhaps the extptr is getting corrupted and pointing to random memory?).

@noamross

This comment has been minimized.

noamross commented Jul 5, 2014

Example (Run in RStudio:

library(httr)
replicate(100, GET("www.noamross.net"))
# Now cancel the command via Ctrl-C or the "Stop" button in RStudio
GET("www.noamross.net")
Error in function (type, msg, asError = TRUE)  : 
  easy handled already used in multi handle
> sessionInfo()  #RStudio 0.98.932
R version 3.0.2 (2013-09-25)
Platform: x86_64-apple-darwin10.8.0 (64-bit)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] httr_0.3

loaded via a namespace (and not attached):
[1] packrat_0.2.0.130 RCurl_1.95-4.1    stringr_0.6.2     tools_3.0.2      
@hadley

This comment has been minimized.

Member

hadley commented Jul 5, 2014

Thanks for the reproducible bug report. I'm not sure what's going wrong here - I think it might be an RCurl bug, but I'm having trouble tracking down the problem.

@hadley hadley closed this in 508a14e Jul 5, 2014

@hadley

This comment has been minimized.

Member

hadley commented Jul 5, 2014

Ok, I think I figured it out - I was incorrectly resetting the handle in between requests. I'm pretty sure it's fixed now.

@noamross

This comment has been minimized.

noamross commented Jul 5, 2014

Just installed httr 3.0.99 via github, but the error persists:

> require(httr)
> replicate(100, GET("www.noamross.net"))
# Pressed "Stop" button here.
> GET("www.noamross.net")
Error in function (type, msg, asError = TRUE)  : 
  easy handled already used in multi handle
> sessionInfo()
R version 3.0.2 (2013-09-25)
Platform: x86_64-apple-darwin10.8.0 (64-bit)

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] httr_0.3.0.99

loaded via a namespace (and not attached):
[1] packrat_0.2.0.130 RCurl_1.95-4.1    stringr_0.6.2     tools_3.0.2 
@hadley

This comment has been minimized.

Member

hadley commented Jul 6, 2014

Hmmm, works for me, if I give it a properly formed url:

replicate(100, GET("http://www.noamross.net"))
# Pressed "Stop" button here.
GET("http://www.noamross.net")
@amirkav

This comment has been minimized.

amirkav commented Oct 13, 2014

Thanks for the very useful package!

I still get this error when my API calls get interrupted (e.g., when intercepting a loop), with the error:
"easy handled already used in multi handle"

I was able to rewrite my API calls using POST(handle=handle(url=...)) which basically creates a new handle for each API call, but still would like to use httr package.

Is there any way to purge all existing handles before making a new API call (or at least when getting the error above)?

Thanks in advance for your help!

Amir

@MarkEdmondson1234

This comment has been minimized.

MarkEdmondson1234 commented Oct 15, 2014

I got this today with this code:

library(httr)
urls <- read.csv("a_list_of_urls.csv", stringsAsFactors=FALSE)
crawl <- lapply(urls, httr::GET)

 # woops a big list of urls, push STOP
 # try again with just one
crawl <- lapply(urls[1], httr::GET)

# the error
@sckott

This comment has been minimized.

sckott commented Nov 25, 2014

@hadley hmmm, I this still doesn't work for me, and apparently others as above.

To make this work, I played around with the idea of internally letting users of our packages set a new handle, but that new handle doesn't seem to be saved in the handle pool:

# run, then esc to cause multi handle error
replicate(50, GET("http://www.noamross.net/"))
# then retry single call, which trows multi handle error
GET("http://www.noamross.net/")
# passing url through handle param instead of url param works again
GET(handle = handle("http://www.noamross.net/"))
# but then another call without passing url through handle throws error again 
GET("http://www.noamross.net/")

Thus, I don't know if this is a no-no, but what if the handle_pool could be exported from httr? That way a user (or the package they are using) can manually delete handles. E.g.

del_handle <- function(...){
  out <- tryCatch(rm(..., envir = httr:::handle_pool), warning = function(w) w)
  if(is(out, "warning")) sprintf("%s does not exist", x) else sprintf("%s removed", x)
}

# run, then esc to cause multi handle error
replicate(50, GET("http://www.noamross.net/"))
# then retry single call, which trows multi handle error
GET("http://www.noamross.net/")
# delete handle
del_handle("http://www.noamross.net/")
# now it works
GET("http://www.noamross.net/")

Using httr_0.5.0.9000

@hadley hadley reopened this Nov 25, 2014

@briatte

This comment has been minimized.

briatte commented Nov 29, 2014

I'm getting the same issue as described above. Using

GET(handle = handle("http://www.example.net/"))

solved the issue for some time, and then crashed, with RStudio reporting the following message when I tried to interrupt the process by pressing Esc:

(TypeError) 
 stack: Bad@http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:5877:26
dqf@http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:7916:51
zad@http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:8242:111
GF@http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:7206:137
WF@http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:8720:228420
http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:360:67
Yk@http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:258:34
bl@http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:493:59
http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html:396:47
 line: 5877
 sourceURL: http://127.0.0.1:11145/rstudio/588422ED93E78B6BAF9BDD676561D32C.cache.html
 __gwt$exception: <skipped>: 'null' is not an object (evaluating 'c[a]')

Also using httr_0.5.0.9000, with R version 3.1.1 and RStudio 0.98.1073.

The same errors pops up after five minutes or so – httr basically seems to 'choke' after too much effort. In my case, it chokes after 800 or so requests on different URLs. Force-quitting RStudio and starting the loop again actually solves the issue – until it crashes again.

@hadley

This comment has been minimized.

Member

hadley commented Dec 4, 2014

Hmmmm, this is a strange problem - I had thought you'd be able to fix it with an explicit reset:

RCurl::reset(handle_url(url = "http://www.noamross.net/")$handle$handle)

That calls curl_easy_reset, which says "this puts back the handle to the same state as it was in when it was just created with curl_easy_init". But it obviously doesn't.

For this version, I'll add a way to reset a handle from the pool, and in the next hopefully fix for good as part of #172

@hadley hadley closed this in 7bd14cf Dec 4, 2014

@aronlindberg

This comment has been minimized.

aronlindberg commented Feb 27, 2015

I am using rgithub and I have httr 0.6.1. loaded (which rgithub depends on). This seems to generate this error again. Is there a newer version to use, or a workaround?

@overvolting

This comment has been minimized.

overvolting commented Mar 15, 2015

I can confirm I've just experienced the same error from the cran version.

@1beb

This comment has been minimized.

Contributor

1beb commented Mar 31, 2015

+1 for this error.

@Ironholds

This comment has been minimized.

Contributor

Ironholds commented Apr 12, 2015

+1 (and a note that it's infected rvest)

@neverfox

This comment has been minimized.

neverfox commented Apr 29, 2015

Same here when calling GET from within a function passed to sapply if I kill the job and try again. Works fine the first time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment