PHP, X-Sendfile and Content-Length

[danielniccoli]

Original author: alobbs (July 12, 2011 09:15:15)

http://lists.octality.com/pipermail/cherokee/2011-June/014903.html

After switching to X-Sendfile, everything looked good. However, we
received notice that some downloads are still being truncated every once
in a while. We have not been able to pinpoint the cause yet, but we have
only seen it happen with IE 8 (other versions could be affected too).

file.php produces 4 headers:

Content-Type: application/force-download
Content-Length: filesize($filepath)
Content-Disposition: attachment; filename="$filename"
X-Sendfile: $filepath

Sniffing the traffic between pound and cherokee revealed that
Content-Length is NOT being sent out by cherokee. Thus, the
client/browser does NOT know about the total filesize and cannot even
detect if the file was completely transferred or not.

Also, the transfer is gzipped (Content-Encoding: gzip) and cherokees
access.log reports differing sizes for the same file. Could that have to

do with gzip?

Original issue: http://code.google.com/p/cherokee/issues/detail?id=1228

[danielniccoli]

From rng.hart...@gmail.com on July 12, 2011 11:37:18
Here are the complete HTTP headers of pound talking to cherokee on localhost:80.
I replaced the filename, domain and ip for privacy reasons.

GET /file.php?file=complete_filename HTTP/1.1
Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, /
Accept-Language: de
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; AskTbARS/5.9.1.14019)
Accept-Encoding: gzip, deflate
Host: example.org
Connection: Keep-Alive
X-SSL-cipher: RC4-MD5 SSLv3 Kx=RSA Au=RSA Enc=RC4(128) Mac=MD5
X-Forwarded-For: 1.2.3.4

HTTP/1.1 200 OK
Connection: close
Date: Wed, 29 Jun 2011 08:01:21 GMT
Server: Cherokee
Content-Encoding: gzip
Vary: Accept-Encoding
ETag: "4e047f98=2daa10"
Last-Modified: Fri, 24 Jun 2011 12:14:16 GMT
Content-Type: application/force-download
Content-Disposition: attachment; filename="complete_filename"

The complete_filename indeed does not have an extension as it is an encrypted/binary file.

[danielniccoli]

From daniel@d15.biz on October 13, 2011 05:50:18
I have noticed the same thing. If you do the exact same request without the "Accept-Encoding: gzip, deflate" header, the content-length header is being sent successfully. You can turn off GZIP (changing "GZip support" on the behavior to "Leave unset") to work around this.

As a workaround, perhaps add a separate behaviour rule for /file.php to forbid GZip. I didn't test this, but it should keep GZip enabled for all other pages, and only disable it for file.php

[skinkie]

The following is actually the problem here. In a few occasions content-length should not be set by the handler. https://github.com/cherokee/webserver/blob/master/cherokee/handler_cgi_base.c#L1083

An example is when we are using gzip, because with gzip enabled the content-length (should become) shorter than without. https://github.com/cherokee/webserver/blob/master/cherokee/connection.c#L1413

I envision the following solution here: X-Sendfile takes priority over content encoding. In some sense thing will result in the fact that X-Sendfile based replies are not encoded. My guess is that if you know what you are doing, and use X-Sendfile, you don't want your webserver to smartass on what you are sending.