error when reading response headers: small read buffer

[fredcy]

If I run muffet http://www.flickr.com/issf2018 I get this output:

error when reading response headers: small read buffer. Increase ReadBufferSize. Buffer size=4096, contents: "HTTP/1.1 302 Found\r\nDate: Fri, 24 Aug 2018 15:29:52 GMT\r\nContent-Type: text/html; charset=UTF-8\r\nContent-Length: 0\r\nP3p: policyref=\"https://policies.yahoo.com/w3c/p3p.xml\", CP=\"CAO DSP COR CUR ADM DEV"...".com https://*.maps.api.here.com https://*.maps.cit.api.here.com https://*.ads.yahoo.com https://cdn.siftscience.com; connect-src https://*.flickr.com https://*.flickr.net http://*.flickr.net https://"

[raviqqe]

It's because buffer size is constant and preconfigured. But I don't know that big header size is common or not.

References

• fasthttp - Client

[splitbrain]

The error also pops up for paypal.com It's a bit annoying when each of your pages has a link to paypal ;-)

[ruzickap]

Any chance to get it fixed?
Looks like muffet http://www.flickr.com/issf2018 still gives the error...

[raviqqe]

A new option for this is added on master:

-b, --buffer-size <size>          Set HTTP body buffer size in bytes. [default: 4096]

Thanks for reporting the issue, all!