Curl Adapter + Proxy Server + curl < 7.30 - Header length incorrect

[wiese]

As described in the curl repo, libcurl prior to 7.30 has a problem correctly determining the header length when a proxy server was involved.
This can lead to problems removing the Transfer-Encoding header.

Other notable projects had similar issues:

• CURLINFO_HEADER_SIZE not supported for curl < 7.30.0 paypal/PayPal-PHP-SDK#258
• Curl proxy header size, wrong header position determined with encoding gzip kriswallsmith/Buzz#181

This can be worked around by

• separating headers and body by \r\n\r\n (a blank line) and using the resulting parts
• calculating the header length differently, cp. $pos = strlen($raw) - curl_getinfo($curl, CURLINFO_SIZE_DOWNLOAD);

or, alternatively, the incompatibility should be documented.

[kevinpapst]

I ran into the exact same problem a couple of days ago.
Doing a call using the Curl Adapter and dumping the response (around line 435), I get:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Date: Mon, 21 Sep 2015 11:49:04 GMT
Expires: Mon, 21 Sep 2015 11:49:34 GMT
Cache-Control: public, max-age=30
Content-Encoding: gzip
Server: mafe
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 443:quic,p=1
Alt-Svc: quic=":443"; p="1"; ma=604800
Connection: close
Transfer-Encoding: chunked

▼8\000\000\000\000\000☻ÿÝ[ßVâJ³¿ßOÁòê;k͞☺¶ÁsCUw1`Dd►186Є↑B !bDæUÎõy81ó\"...

Then dumping the modified $responseHeaders I get:

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Date: Mon, 21 Sep 2015 11:49:04 GMT
Expires: Mon, 21 Sep 2015 11:49:34 GMT
Cache-Control: public, max-age=30
Content-Encoding: gzip
Server: mafe
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 443:quic,p=1
Alt-Svc: quic=":443"; p="1"; ma=604800
Connection: close

But after the substr_replace in https://github.com/zendframework/zend-http/blob/master/src/Client/Adapter/Curl.php#L454 the header is back to its original state (due to the wrong header size calculation), leaving the Transfer-Encoding: chunked as part of the HTTP response.

That header is found in https://github.com/zendframework/zend-http/blob/master/src/Response.php#L340 leading to the problem, that the Response object tries to decode the body as chunked (which has already been done by libcurl) then failing and throwing a RuntimeException "Error parsing body - doesn't seem to be a chunked message".

+1 for implementing a different header length calculation, as the buggy libcurl version is shipped in major distros (e.g. CentOS 7)

[arnaud-lb]

+1 for implementing a different header length calculation, as the buggy libcurl version is shipped in major distros (e.g. CentOS 7)

Guzzle seems to be using CURLOPT_HEADERFUNCTION / CURLOPT_WRITEFUNCTION in order to receive the header and the body separately: https://github.com/guzzle/guzzle/blob/b65cdda6b0d237f7a47c25414599c5a400fc4b83/src/Handler/CurlFactory.php#L53

This avoids the need from parsing / calculating positions

[michalbundyra]

Documentation updated. Thanks, @wiese!