Limiting the number of downloading bytes

[dmitry]

After reading through the source I haven't found the option that gives ability to set the maximum number of bytes, that allowed to be downloaded in a request. In my opinion this feature might be somewhere in dump_chunks, but please lets discuss about the details and whether this functionality might be implemented in this gem at all.

[nahi]

No, it doesn't have that feature. I think the feature would be useful as an API client.

The limit is for response, not request, right? So the extension points would be around read_body_length, read_body_chunked and read_body_rest in session.rb.

[dmitry]

Yes, you are right, my mistake - limit is for the response.

• Can I make a PR for this gem or should I make an extension outside of this gem?
• Should it raise an exception in case of size of response is overpassed (exception might be somewhere here: https://github.com/nahi/httpclient/blob/master/lib/httpclient.rb#L270)?
• What else should I think of before I even start to develop a patch or an extension?

One more question:

Why not put maximum length check in more higher level get_body (

httpclient/lib/httpclient.rb

Line 1174 in b225d9e

sess.get_body do |part|

) code?

[nahi]

• Would you please create a PR? I want to merge it as an additional feature (unlimited by default.)
• I think BadResponseError is enough but new error would be fine if we find a good name.
• Please include testcases that covers all lines of additional logic so that we can keep the code healthy in the future.

Because HTTPClient#do_get_* and Session#get_body handles decoded body, not actual HTTP message body. We want to limit the downloading bytes, not un-gzipped bytes, right?