Consider removing header: Content-Length on FrontendPage as it gives all pages a 5 sec delay in some environments #736

Closed
kanduvisla opened this Issue Aug 9, 2011 · 29 comments

Comments

Projects
None yet
9 participants
@kanduvisla
Contributor

kanduvisla commented Aug 9, 2011

I don't know exactly why this is occurring, but on one hosting-environment, rule 27 of index.php is causing issues:

header(sprintf('Content-Length: %d', strlen($output)));

This rule caused my page to load for 80%, then wait for 5 seconds, and then loading the remaining 20%. Commenting this line fixed the issue. But I'm not sure exactly why this was happening. The numbers where correct: the strlen()-function gave the correct number of bytes of the string. :-/

I also tried it by disabling all the extensions I used, but the issue was still occurring. Also the backend of Symphony was affected by the 5-second delay. This also made me wonder: is it really this important to send the Content-Length-header? Or can this be omitted? I could imagine there are more hosting-environments out there which have some strange mumbo-jumbo configuration that could cause an issue with this.

@klaftertief

This comment has been minimized.

Show comment
Hide comment
@klaftertief

klaftertief Aug 9, 2011

Contributor

Are you running PHP with mod_fastcgi? Sounds like http://symphony-cms.com/discuss/thread/40083/2/#position-30.

Contributor

klaftertief commented Aug 9, 2011

Are you running PHP with mod_fastcgi? Sounds like http://symphony-cms.com/discuss/thread/40083/2/#position-30.

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Aug 9, 2011

Contributor

Yes, my hosts' server API is CGI/FastCGI. But it's not that I'm using something like that (or mod_deflate) in my code/.htaccess.

Contributor

kanduvisla commented Aug 9, 2011

Yes, my hosts' server API is CGI/FastCGI. But it's not that I'm using something like that (or mod_deflate) in my code/.htaccess.

@brendo

This comment has been minimized.

Show comment
Hide comment
@brendo

brendo Aug 9, 2011

Member

The same tests @pointybeard mentioned in that thread hold true today. Is your site using multibyte characters? How are you verifying that strlen is returning the correct value?

Member

brendo commented Aug 9, 2011

The same tests @pointybeard mentioned in that thread hold true today. Is your site using multibyte characters? How are you verifying that strlen is returning the correct value?

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Aug 9, 2011

Contributor

I made a test with a raw file, containing the HTML output of the webpage, and compared the number of bytes of that file with the number returned by strlen():

$content = file_get_contents('test'); // 'test' is the name of the file
header(sprintf('Content-Length: %d', strlen($content)));
echo $content;
Contributor

kanduvisla commented Aug 9, 2011

I made a test with a raw file, containing the HTML output of the webpage, and compared the number of bytes of that file with the number returned by strlen():

$content = file_get_contents('test'); // 'test' is the name of the file
header(sprintf('Content-Length: %d', strlen($content)));
echo $content;
@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Aug 9, 2011

Contributor

Hm, if mod_fastcgi seems to be causing this issue (which it most certainly seems it does), I think there should be a check of some kind whether or not to set the Content-Length-header.

Contributor

kanduvisla commented Aug 9, 2011

Hm, if mod_fastcgi seems to be causing this issue (which it most certainly seems it does), I think there should be a check of some kind whether or not to set the Content-Length-header.

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Aug 9, 2011

Contributor

Perhaps something like this instead of line 27 of index.php?

if(php_sapi_name() != 'cgi-fcgi')
{
    header(sprintf('Content-Length: %d', strlen($output)));
}
Contributor

kanduvisla commented Aug 9, 2011

Perhaps something like this instead of line 27 of index.php?

if(php_sapi_name() != 'cgi-fcgi')
{
    header(sprintf('Content-Length: %d', strlen($output)));
}
@brendo

This comment has been minimized.

Show comment
Hide comment
@brendo

brendo Aug 9, 2011

Member

Some other references for this bug, Debian, Chrome

I'm weary of hardcoding a workaround. I'm actually wondering whether it should just be removed altogether. On my particular setup, setting the Content Type has no affect in index.php as Apache will override it with the gzipped length. This doesn't seem to be the case on all servers though.

Member

brendo commented Aug 9, 2011

Some other references for this bug, Debian, Chrome

I'm weary of hardcoding a workaround. I'm actually wondering whether it should just be removed altogether. On my particular setup, setting the Content Type has no affect in index.php as Apache will override it with the gzipped length. This doesn't seem to be the case on all servers though.

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Aug 9, 2011

Contributor

+1 for removing the header alltogether

Contributor

kanduvisla commented Aug 9, 2011

+1 for removing the header alltogether

@michael-e

This comment has been minimized.

Show comment
Hide comment
@michael-e

michael-e Aug 9, 2011

Member

I vote for being extra careful in this case. Removing the header might have side effects, especially if the server does not overwrite it (with the gzipped length, for example).

I will make some tests on Debian and report back.

Member

michael-e commented Aug 9, 2011

I vote for being extra careful in this case. Removing the header might have side effects, especially if the server does not overwrite it (with the gzipped length, for example).

I will make some tests on Debian and report back.

@michael-e

This comment has been minimized.

Show comment
Hide comment
@michael-e

michael-e Aug 9, 2011

Member

I did some tests with Apache 2.2.16 on a Debian Squeeze server.

When a GET request is received (which is what a browser sends), Apache will always add the Content-Length header (if it has not been created by Symphony). This is true even if the content is served without being gzipped.

I assume that any web server will behave in the same way — adding headers which are useful. So I have nothing against @brendo's and @kanduvisla's proposal to remove that line.

[EDIT]: Deleted what I said about HEAD requests. It was nonsense...

Member

michael-e commented Aug 9, 2011

I did some tests with Apache 2.2.16 on a Debian Squeeze server.

When a GET request is received (which is what a browser sends), Apache will always add the Content-Length header (if it has not been created by Symphony). This is true even if the content is served without being gzipped.

I assume that any web server will behave in the same way — adding headers which are useful. So I have nothing against @brendo's and @kanduvisla's proposal to remove that line.

[EDIT]: Deleted what I said about HEAD requests. It was nonsense...

@brendo

This comment has been minimized.

Show comment
Hide comment
@brendo

brendo Aug 10, 2011

Member

I guess what we need a surefire way to test this across a couple of environments! I know omitting a Content-Length header altogether is bad practice, but if we can be sure that Apache automatically does this, then I have no qualms about removing it.

What we need to be cautious of is removing it to fix the 'minority' of cases, only to have it produce undesirable results in a different 'minority'.

@kanduvisla, does your server automatically add the Content-Length header if it's not generated by Symphony?

Member

brendo commented Aug 10, 2011

I guess what we need a surefire way to test this across a couple of environments! I know omitting a Content-Length header altogether is bad practice, but if we can be sure that Apache automatically does this, then I have no qualms about removing it.

What we need to be cautious of is removing it to fix the 'minority' of cases, only to have it produce undesirable results in a different 'minority'.

@kanduvisla, does your server automatically add the Content-Length header if it's not generated by Symphony?

@brendo

This comment has been minimized.

Show comment
Hide comment
@brendo

brendo Aug 10, 2011

Member

Some tests from me:

Apache/2.2.8 on Ubuntu, sends Content Length even if omitted (gzip is enabled at the server level)

Apache/2.2.17 on Unix, sends Content Length even if omitted (gzip is enabled at the server level).

Apache/2.2.17 on FreeBSD, sends Content Length even if omitted (gzip is enabled at a site level). If gzip is turned off, the Content Length header is not automatically added.

@michael-e does the result change if you turn off gzip for the site?

Member

brendo commented Aug 10, 2011

Some tests from me:

Apache/2.2.8 on Ubuntu, sends Content Length even if omitted (gzip is enabled at the server level)

Apache/2.2.17 on Unix, sends Content Length even if omitted (gzip is enabled at the server level).

Apache/2.2.17 on FreeBSD, sends Content Length even if omitted (gzip is enabled at a site level). If gzip is turned off, the Content Length header is not automatically added.

@michael-e does the result change if you turn off gzip for the site?

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Aug 10, 2011

Contributor

@brendo: No. You can check the response headers on http://www.thecarethinktank.com/ . phpinfo: http://www.thecarethinktank.com/info.php

Could you please let me know when you're done checking, then I can delete the info.php-file, since I don't like it that anyone can see the server configuration.

I have no control over the server settings, since we use an external company to host our websites.

I checked another configuration of a site that doesn't encounter the 5 second lag because of the Content-Length: http://www.hydrolysates.com/ . phpinfo: http://www.hydrolysates.com/info.php

Contributor

kanduvisla commented Aug 10, 2011

@brendo: No. You can check the response headers on http://www.thecarethinktank.com/ . phpinfo: http://www.thecarethinktank.com/info.php

Could you please let me know when you're done checking, then I can delete the info.php-file, since I don't like it that anyone can see the server configuration.

I have no control over the server settings, since we use an external company to host our websites.

I checked another configuration of a site that doesn't encounter the 5 second lag because of the Content-Length: http://www.hydrolysates.com/ . phpinfo: http://www.hydrolysates.com/info.php

@brendo

This comment has been minimized.

Show comment
Hide comment
@brendo

brendo Aug 10, 2011

Member

You can remove those files now, I've saved them locally, thanks.

edit
http://www.thecarethinktank.com/ is sent gzipped and has no Content Length header

http://www.hydrolysates.com/ is not gzipped and has a Content Length header

Do these sites have the Content Length header still being set by Symphony? Or has it been removed to test?

Member

brendo commented Aug 10, 2011

You can remove those files now, I've saved them locally, thanks.

edit
http://www.thecarethinktank.com/ is sent gzipped and has no Content Length header

http://www.hydrolysates.com/ is not gzipped and has a Content Length header

Do these sites have the Content Length header still being set by Symphony? Or has it been removed to test?

@michael-e

This comment has been minimized.

Show comment
Hide comment
@michael-e

michael-e Aug 10, 2011

Member

@brendo: I experienced the same behaviour with or without gzip (being enabled at server level, i.e. using mod_deflate).

I talked to a friend of mine, and he said that the behaviour will depened on the Apache configuration. It might as well serve the content with a Transfer-Encoding: chunked header — in this case the content length is not needed. Or (if the client speaks HTTP 1.0 only) Apache may even decide to send the content without Content-Length and simply drop the connection when it's finished (which also tells the client to not wait anymore).

Can you double-check the FreeBSD server?

Member

michael-e commented Aug 10, 2011

@brendo: I experienced the same behaviour with or without gzip (being enabled at server level, i.e. using mod_deflate).

I talked to a friend of mine, and he said that the behaviour will depened on the Apache configuration. It might as well serve the content with a Transfer-Encoding: chunked header — in this case the content length is not needed. Or (if the client speaks HTTP 1.0 only) Apache may even decide to send the content without Content-Length and simply drop the connection when it's finished (which also tells the client to not wait anymore).

Can you double-check the FreeBSD server?

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Aug 10, 2011

Contributor

@brendo: for www.thecarethinktank.com I removed the Content-Length header, since it is live, but that was the site giving the issues with the 5 sec delay. www.hydrolysates.com still gets the Content-Length header sent by Symphony, but since it's not gZipped it makes sense, because as mentioned before, it probably has to do with the bug that Content-Length doesn't get adjusted on some servers after gZipping.

Contributor

kanduvisla commented Aug 10, 2011

@brendo: for www.thecarethinktank.com I removed the Content-Length header, since it is live, but that was the site giving the issues with the 5 sec delay. www.hydrolysates.com still gets the Content-Length header sent by Symphony, but since it's not gZipped it makes sense, because as mentioned before, it probably has to do with the bug that Content-Length doesn't get adjusted on some servers after gZipping.

@brendo

This comment has been minimized.

Show comment
Hide comment
@brendo

brendo Aug 12, 2011

Member

I found this and this. I think my FreeBSD server is using chunked encoding, so it's not required to set a Content Length.

Member

brendo commented Aug 12, 2011

I found this and this. I think my FreeBSD server is using chunked encoding, so it's not required to set a Content Length.

@michael-e

This comment has been minimized.

Show comment
Hide comment
@michael-e

michael-e Aug 12, 2011

Member

So shall we assume that the server will do it right? I tend towards saying "let's try it and see how it behaves in the wild".

Member

michael-e commented Aug 12, 2011

So shall we assume that the server will do it right? I tend towards saying "let's try it and see how it behaves in the wild".

@nilshoerrmann

This comment has been minimized.

Show comment
Hide comment
@nilshoerrmann

nilshoerrmann Aug 12, 2011

Member

I tend towards saying "let's try it and see how it behaves in the wild".

Michael, is everything fine with you? That really doesn't sound like you :o)

Member

nilshoerrmann commented Aug 12, 2011

I tend towards saying "let's try it and see how it behaves in the wild".

Michael, is everything fine with you? That really doesn't sound like you :o)

@michael-e

This comment has been minimized.

Show comment
Hide comment
@michael-e

michael-e Aug 12, 2011

Member

Well, as long as the force is with us...

(As you see, the Jedi school really improved my point of view.)

:-)

Member

michael-e commented Aug 12, 2011

Well, as long as the force is with us...

(As you see, the Jedi school really improved my point of view.)

:-)

@nitriques

This comment has been minimized.

Show comment
Hide comment
@nitriques

nitriques Aug 14, 2011

Member

I am having troubles with the content-length in Safari now. If it is not present, sometimes safari will just display a blank page... If it is present (but has the wrong value) it adds a nasty delay to the page load....

Yes, gzip is activated on my server and multibytes-strings too... I am not running with FastCGI, but with suExec.

Member

nitriques commented Aug 14, 2011

I am having troubles with the content-length in Safari now. If it is not present, sometimes safari will just display a blank page... If it is present (but has the wrong value) it adds a nasty delay to the page load....

Yes, gzip is activated on my server and multibytes-strings too... I am not running with FastCGI, but with suExec.

@nickdunn nickdunn closed this Aug 19, 2011

@nickdunn nickdunn reopened this Aug 19, 2011

@nickdunn nickdunn closed this Aug 19, 2011

@nickdunn nickdunn reopened this Aug 19, 2011

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Sep 23, 2011

Contributor

I had this issue today again on another host. Simply removing the line made the load time of each page 3 sec. faster!

Please, please, PLEASE! Remove this line in the next Symphony release since I see no benefits, only drawbacks of it being present.

Contributor

kanduvisla commented Sep 23, 2011

I had this issue today again on another host. Simply removing the line made the load time of each page 3 sec. faster!

Please, please, PLEASE! Remove this line in the next Symphony release since I see no benefits, only drawbacks of it being present.

@michael-e

This comment has been minimized.

Show comment
Hide comment
@michael-e

michael-e Sep 23, 2011

Member

No way. You must update your Symposium 2011 status first. :-)

Member

michael-e commented Sep 23, 2011

No way. You must update your Symposium 2011 status first. :-)

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Sep 23, 2011

Contributor

Done :-P

offtopic: I'm sorry guys, I really wanted to come, but can't fit it in with my work etc. this year. Perhaps next year... :-(

Contributor

kanduvisla commented Sep 23, 2011

Done :-P

offtopic: I'm sorry guys, I really wanted to come, but can't fit it in with my work etc. this year. Perhaps next year... :-(

@michael-e

This comment has been minimized.

Show comment
Hide comment
@michael-e

michael-e Sep 23, 2011

Member

What a shame. But thanks for updating!

Member

michael-e commented Sep 23, 2011

What a shame. But thanks for updating!

@remie

This comment has been minimized.

Show comment
Hide comment
@remie

remie Sep 23, 2011

Contributor

From the HTTP specifications, section 14.13

In HTTP, it ('Content-Length', remie) SHOULD be sent whenever the message's length can be determined prior to being transferred, unless this is prohibited by the rules in section 4.4.

Content-Length is only optional in case the transfer-encoding is chuncked.

Section 4.4 states

Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non- identity transfer-coding, the Content-Length MUST be ignored.

Basically, I think it is safe to assume that the content-length will be set by any web server that is HTTP/1.1 compliant. As the documentation states that Symphony requires web server configuration of Apache or Litespeed webserver, and these web servers are know to be HTTP/1.1 compliant, this line can safely be removed.

Contributor

remie commented Sep 23, 2011

From the HTTP specifications, section 14.13

In HTTP, it ('Content-Length', remie) SHOULD be sent whenever the message's length can be determined prior to being transferred, unless this is prohibited by the rules in section 4.4.

Content-Length is only optional in case the transfer-encoding is chuncked.

Section 4.4 states

Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non- identity transfer-coding, the Content-Length MUST be ignored.

Basically, I think it is safe to assume that the content-length will be set by any web server that is HTTP/1.1 compliant. As the documentation states that Symphony requires web server configuration of Apache or Litespeed webserver, and these web servers are know to be HTTP/1.1 compliant, this line can safely be removed.

@kanduvisla

This comment has been minimized.

Show comment
Hide comment
@kanduvisla

kanduvisla Sep 23, 2011

Contributor

Hurray!

Contributor

kanduvisla commented Sep 23, 2011

Hurray!

@kanduvisla kanduvisla closed this Sep 23, 2011

@michael-e michael-e reopened this Sep 23, 2011

@michael-e

This comment has been minimized.

Show comment
Hide comment
@michael-e

michael-e Sep 23, 2011

Member

Wait, it has not been pulled yet!

@brendo: Shall we?

Member

michael-e commented Sep 23, 2011

Wait, it has not been pulled yet!

@brendo: Shall we?

@DavidOliver

This comment has been minimized.

Show comment
Hide comment
@DavidOliver

DavidOliver Dec 15, 2011

Member

I came across this issue because I'd neglected to update Symphony. In case it's desirable or beneficial to send the Content-Length header when possible, here's how it's being handled in the Banshee framework.

libraries/output.php

Heavily snipped relevant portions of function:

    /* Generate output via XSLT
    public function generate($output) {
        switch ($output) {
            case null:
                $php_gzip = ini_get("zlib.output_compression");
                if (is_false($php_gzip)) {
                    header("Content-Length: ".strlen($html));
                }
                print $html;
                break;
        }
    }

Full function for reference:

    /* Generate output via XSLT
     *
     * INPUT:  string output type
     * OUTPUT: -
     * ERROR:  -
     */
    public function generate($output) {
        if ($this->disabled) {
            return;
        }

        if (($output == null) && $this->page->ajax_request) {
            $output = "xml";
        }

        switch ($output) {
            case "json":
                $data = $this->array;
                $data = $this->optimize_for_json($data);
                header("Content-Type: application/json");
                print json_encode($data["output"]);
                break;
            case "xml":
                header("Content-Type: text/xml");
                print $this->document;
                break;
            case "data":
                header("Content-Type: text/plain");
                print $this->document;
                break;
            case null:
                if (($html = parent::transform($this->page->view)) === false) {
                    print "XSL Transformation error";
                    return;
                }

                /* GZip content encoding
                 */
                $encodings = $_SERVER["HTTP_ACCEPT_ENCODING"];
                $php_gzip = ini_get("zlib.output_compression");
                if (($encodings !== null) && (strlen($html) >= 1024) && is_false($php_gzip)) {
                    $encodings = explode(",", $encodings);
                    foreach ($encodings as $encoding) {
                        $encoding = trim($encoding);
                        if ($encoding == "gzip") {
                            header("Content-Encoding: gzip");
                            $html = gzencode($html, 6);
                            break;
                        }
                    }
                }

                /* Print output
                 */
                header("Content-Type: ".$this->content_type);
                header("Content-Language: ".$this->language);
                if (is_false($php_gzip)) {
                    header("Content-Length: ".strlen($html));
                }
                header("X-Powered-By: Banshee PHP framework v".BANSHEE_VERSION);
                print $html;
                break;
            default:
                print "Unknown output type";
        }
    }
Member

DavidOliver commented Dec 15, 2011

I came across this issue because I'd neglected to update Symphony. In case it's desirable or beneficial to send the Content-Length header when possible, here's how it's being handled in the Banshee framework.

libraries/output.php

Heavily snipped relevant portions of function:

    /* Generate output via XSLT
    public function generate($output) {
        switch ($output) {
            case null:
                $php_gzip = ini_get("zlib.output_compression");
                if (is_false($php_gzip)) {
                    header("Content-Length: ".strlen($html));
                }
                print $html;
                break;
        }
    }

Full function for reference:

    /* Generate output via XSLT
     *
     * INPUT:  string output type
     * OUTPUT: -
     * ERROR:  -
     */
    public function generate($output) {
        if ($this->disabled) {
            return;
        }

        if (($output == null) && $this->page->ajax_request) {
            $output = "xml";
        }

        switch ($output) {
            case "json":
                $data = $this->array;
                $data = $this->optimize_for_json($data);
                header("Content-Type: application/json");
                print json_encode($data["output"]);
                break;
            case "xml":
                header("Content-Type: text/xml");
                print $this->document;
                break;
            case "data":
                header("Content-Type: text/plain");
                print $this->document;
                break;
            case null:
                if (($html = parent::transform($this->page->view)) === false) {
                    print "XSL Transformation error";
                    return;
                }

                /* GZip content encoding
                 */
                $encodings = $_SERVER["HTTP_ACCEPT_ENCODING"];
                $php_gzip = ini_get("zlib.output_compression");
                if (($encodings !== null) && (strlen($html) >= 1024) && is_false($php_gzip)) {
                    $encodings = explode(",", $encodings);
                    foreach ($encodings as $encoding) {
                        $encoding = trim($encoding);
                        if ($encoding == "gzip") {
                            header("Content-Encoding: gzip");
                            $html = gzencode($html, 6);
                            break;
                        }
                    }
                }

                /* Print output
                 */
                header("Content-Type: ".$this->content_type);
                header("Content-Language: ".$this->language);
                if (is_false($php_gzip)) {
                    header("Content-Length: ".strlen($html));
                }
                header("X-Powered-By: Banshee PHP framework v".BANSHEE_VERSION);
                print $html;
                break;
            default:
                print "Unknown output type";
        }
    }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment