Is there an option to close a connection following use?

[timwilliate]

After successfully making GET requests to a private API, I noticed the following text at the end of the (verbose) output:

Connection #0 to host {host_name} left intact

I can see from the httr documentation that by default, connections are left open and then reused. Is there a method which can be used to explicitly close all open connections? I would like to perform such an operation after executing a series of GET requests in order to ensure that no http connections are left open on the server.

Thank you

[hadley]

Could you give me a little more detail about your requirements? It normally doesn't matter if the connection is left open because it will get closed automatically after a reasonable timeout.

(If you really want to do this you'll have to manually manage the curl handles instead of depending on httr's global connection pool)

[timwilliate]

Thank you for responding so quickly!

In this particular scenario, the length of the timeout is what is tripping up my application. I am utilizing GET requests inside analysis software that is running across several hundred CPU's. The authentication server that I am required to hit is becoming overwhelmed with the number of connections left open, and thus performance is suffering greatly (it is almost like having a memory leak).

That is what led me to the question. If it is possible to close large blocks of connections when I no longer need them, this issue is avoided. Alternatively, is it possible to at least control the length of the timeout?

Thank you very much for getting back with me.

[hadley]

One option is to create an explicit handle. When you delete it, curl will close the connection automatically.

my_handle <- handle("http://my-server/")
GET(handle = my_handle, path = "index.html")
GET(handle = my_handle, path = "index.html")
rm(my_handle)

You could also do:

GET(..., config(forbid.reuse = 1))

But that's likely to have equally bad performance implications because it will force the server to open and close a connection for every request.

[timwilliate]

I am attempting you utilize your strategy on explicit handle creation. When I attempt to create a handle using the code below (the cookie is a required input scraped off an earlier request)

h <- handle("http://my-server.com/")
GET(handle=h, path = "id/query", add_headers(Cookie = c))

I receive the following error message, which confuses me, because I am not sure what it means by "only one url or handle".

Error in handle_url(handle, url, ...) : 
  Must specify exactly one of url or handle

Thank you for your advice.

[hadley]

Oh hmm, I guess I didn't think that change through - I think you can work around it by setting url = NULL in the call.

[timwilliate]

Using url=NULL worked. Thank you very much for your help. I think you have done some great things with this library. I originally worked this problem space in Python (where the 'Requests' module makes http authentication really simple. Your library is the best such http library I have used in R thus far.

[hadley]

Awesome :) Do let me know if there's stuff you think is substantially easier in requests. The audience (and hence goals) of httr is somewhat different to requests, but most things should be as easy.

[hadley]

I'll be submitting the next version of httr (0.4) to cran tomorrow :)

[datafj]

You could also do:

GET(..., config(forbid.reuse = 1))

Actually it is GET(..., config(forbid_reuse = 1)), with an underscore between forbid and reuse, not a period.