Request: Allow setting of options of underlying http libraries

[ljckennedy]

Some of the libraries used by Faraday support attributes that seem to be unavailable in Faraday (e.g. chunk_size, NTLM Authenitcation.

Can we have a way to set these?

There may already bee a way to do this, but If there is I can't find documentation and/or examples.

[iMacTia]

Hi @ljckennedy I'm not really sure what do you mean in this case for "libraries".
Do you mean the adapters? Would you be able to provide a quick example of what do you need to do?

[ljckennedy]

Yes - Adapters. Apologies for being vague. For example I want to set the chunksize for mulitpart downloads and to use NTLM for Authentication.
( http://www.rubydoc.info/gems/httpclient/2.8.2.4/HTTPClient%2FSessionManager%3Achunk_size & http://dev.scottw.com/accessing-restful-service-ruby-via )

2 separate problems. With the former I have having trouble downloading large files (~2gb) and suspect this may help. The later is requirements for some APIs I need to call.

Thanks.

[iMacTia]

I see, thanks for the clarification.
This feature was only added in the latest version of the gem (0.11.0) so for this reason documentation is still missing (there are a lot of things missing, so I'm planning to do a Wiki).
This is what you can do:

conn = Faraday.new(...) do |f|
  f.adapter :httpclient do |client| # yields HTTPClient
    client.chunk_size = CHUNK_SIZE
    client.set_auth(domain, user, password)
  end
end

I'm not really sure about the chunk_size, as I couldn't find how to set it, however keep in mind that client is an instance of HTTPClient

[iMacTia]

Just added a new section to the Readme until the Wiki is done.
See https://github.com/lostisland/faraday/blob/master/README.md#ad-hoc-adapters-customization

[ljckennedy]

Thanks that's awesome !

BTW: Am I write in assuming that if I have set conn.request :multipart that faraday will chunk the download? Or do I need code to handle receiving the chunks. I could not find an example so I assumed it is built in, but thought i'd get confirmation.

Thanks again.

[ljckennedy]

So I played with the chunk_size today and it isn't helping with my issue, so maybe I was on the wrong track. Realise this might not be the ideal place to ask, but do you have an example of how to download a very large file? I can download a 600mb file in a minute (from my local network) but a 2.4gb file never finishes. I guess just using multipart and response = conn.get() isn't enough? Do you have an example of how I should do this?

[iMacTia]

Hi @ljckennedy I think what you're actually interested in is something called "streaming request".
When you perform a streaming request, the data is not downloaded all in one shot, but split into chunks of data.
This is usually done passing a callback to most of the HTTP libraries.
Unfortunately, Faraday doesn't support streaming yet, but it's a planned feature for v1.0.
See also #620 #604

I'll close this issue simply because I believe the original question "Allow setting of options of underlying http libraries" was answered, but please feel free to reopen if that's not the case.