URL with query strings cached even when GET Request caching disabled #216

Closed
raamdev opened this Issue Jun 16, 2014 · 41 comments

Projects

None yet

4 participants

@raamdev
Member
raamdev commented Jun 16, 2014

Specifically RSS feed links that have query string variables. Needs testing. I haven't reproduced this yet.

Reported here:

http://wordpress.org/support/topic/url-with-query-strings-cached-even-though-option-is-disabled

@raamdev raamdev added this to the Next Release milestone Jun 16, 2014
@cnxsoft
cnxsoft commented Jun 17, 2014

Just to clarify, it's not only RSS, but any links with query strings. For example:

grep "Quick Cache file built" *
0090e43a2a613c00d8372f62214905af.html:
03230502662519e27fddc5fd3b554b15.html:
0409297ff39f99b466bf80fd59a74cf0.html:
2269f6abb7216390e6e88eb75f540f32.html:
8a0eabb8c405e44f8446bfe86ad38573.html:
a379542b498725d7e4fcccf1c1cd42e2.html:
c21a892a88d0583e7f47b68ffe423063.html:
ca664ccb4d47c684f4b01655540d3bce.html:
e65e7ed9b9bec2eecee3351dc96a6858.html:

@jaswrks
Member
jaswrks commented Jun 17, 2014

I have been unable to reproduce this so far.
See: http://jason.websharks-inc.net/sample-page/?q=v


        </body>
</html>
<!-- Quick Cache is NOT caching this page, because `$_GET` contains query string data. The current configuration says NOT to cache GET requests with a query string. -->

@cnxsoft For you to have this issue it would make me think that either you have GET request caching enabled with Quick Cache? Or, you have a theme/plugin that is somehow altering PHP's super globals array to make other applications believe there is in fact no query string in the request.

See also this line in the source code for a reference.

The easiest way to troubleshoot this is to make an attempt to reproduce the bug in a fresh/clean installation of WordPress. One that is running one of the default WP themes, and with only the Quick Cache plugin active. Are you able to reproduce this problem in that environment as well?

@raamdev
Member
raamdev commented Jun 18, 2014

Punting this to the Future Release milestone until we get more feedback or find a way to reproduce this.

@raamdev raamdev modified the milestone: Future Release, Next Release Jun 18, 2014
@cnxsoft
cnxsoft commented Jun 18, 2014

Ok. I'll need to try that when I have time.. So it may take two weeks or so.

@cnxsoft
cnxsoft commented Jun 28, 2014

I can confirm I can reproduce the issue in a Wordpress 3.8 installation in a different machine with Quick Cache being the only plugin, and the selected theme "Twenty Twelve".
I'm using Ubuntu 14.04
nginx/1.4.6
php5-fpm - PHP 5.5.9-1ubuntu4.2
mysql 5.5.37-0ubuntu0.14.04.1

With the index page: http://localhost:8080/ and http://localhost:8080/?q=v, no problem.
But with post, I'll have extra cached file in "post".q directory

@raamdev
Member
raamdev commented Jun 29, 2014

@cnxsoft Thank you very much. I will setup a test environment within the next few days using the same software and versions and attempt to reproduce this bug.

@jaswrks
Member
jaswrks commented Jun 29, 2014

With the index page: http://localhost:8080/ and http://localhost:8080/?q=v, no problem.
But with post, I'll have extra cached file in "post".q directory

How that is happening without there being a larger server issue is beyond me. I'd love to see a dump of $_SERVER and/or $_GET when this occurs. @cnxsoft If you create the following file and access it like this from that server, what do you get please? If you can paste the results for us I'd love to see them.

http://localhost:8080/test.php?q=v
Create test.php with the following test code.

<?php
header('Content-Type: text/plain; charset=UTF-8');
echo 'GET = '. print_r($_GET, TRUE)."\n\n";
echo 'REQUEST = '. print_r($_REQUEST, TRUE)."\n\n";
echo 'QUERY_STRING = '. print_r($_SERVER['QUERY_STRING'], TRUE)."\n\n";
echo 'QUICK_CACHE_GET_REQUESTS = '.print_r(QUICK_CACHE_GET_REQUESTS, TRUE)."\n\n";

Expected Output...

GET = Array
(
    [q] => v
)


REQUEST = Array
(
    [q] => v
)


QUERY_STRING = q=v

QUICK_CACHE_GET_REQUESTS = QUICK_CACHE_GET_REQUESTS

It would also be great if you could show us the output from a file that contains...

<?php phpinfo(); ?>
@cnxsoft
cnxsoft commented Jun 30, 2014

The output is as expected:
GET = Array
(
[q] => v
)

REQUEST = Array
(
[q] => v
)

QUERY_STRING = q=v

QUICK_CACHE_GET_REQUESTS = QUICK_CACHE_GET_REQUESTS

You can find the output of phpinfo @ http://pastebin.com/H4a87kQn

@jaswrks
Member
jaswrks commented Jun 30, 2014

@cnxsoft Awesome! Thanks :-)

Would you mind also pasting a copy of this file from your WP installation please?
Please paste the contents of this file: /wp-content/advanced-cache.php

@cnxsoft
cnxsoft commented Jun 30, 2014

advanced-cache.php --> http://pastebin.com/fHtmcMy8

@cnxsoft
cnxsoft commented Jun 30, 2014

Since I assume you are going to test with nginx, you might also consider having a look @ http://wordpress.org/support/topic/random-blank-pages-2, which appears to only happen with nginx. It's just much more difficult to reproduce.

@jaswrks
Member
jaswrks commented Jun 30, 2014

@cnxsoft Great! Thank you. Everything seems to be in order there. I am intrigued. We are investigating this, so the information you provided there should help us out in a big way. Appreciate that. I'm sure one of us will update this again very soon. Assuming we can reproduce it.

Since I assume you are going to test with nginx, you might also consider having a look @ http://wordpress.org/support/topic/random-blank-pages-2, which appears to only happen with nginx. It's just much more difficult to reproduce.

I believe this issue was corrected in the current dev branch after our work here: #220 It has not been released yet, but will be pretty soon.

@raamdev
Member
raamdev commented Aug 4, 2014

@cnxsoft I'm looking at this again. Is there any chance you could share the contents of your nginx configuration files? If you'd like to do that privately, you can use this link (expires in 72 hours): http://wsharks.com/1ltP8Tz I'm wondering if maybe this issue is related to a specific nginx configuration.

@cnxsoft
cnxsoft commented Aug 5, 2014

I've attached the config via the link you provided.

@raamdev
Member
raamdev commented Aug 5, 2014

@cnxsoft Thank you. I've received the configuration. I will update here again when I have more details regarding my attempt to reproduce this.

Nginx configuration: https://websharks.zendesk.com/agent/#/tickets/3423

@jaswrks
Member
jaswrks commented Aug 9, 2014

@cnxsoft Here is another test that you could run. It might offer us some further insight. It's actually the same test that you ran before, but this time let's bring your WordPress installation into the mix also. Just to be sure there is not another wildcard at play that we missed in the previous testing.

Please create this file in the root of your WordPress installation; i.e. right along-side your /wp-config.php file, in the same directory where it lives.

/qc-test.php

<?php
require_once dirname(__FILE__).'/wp-load.php';

header('Content-Type: text/plain; charset=UTF-8');

echo 'GET = '. print_r($_GET, TRUE)."\n\n";
echo 'REQUEST = '. print_r($_REQUEST, TRUE)."\n\n";
echo 'QUERY_STRING = '. print_r($_SERVER['QUERY_STRING'], TRUE)."\n\n";
echo 'QUICK_CACHE_GET_REQUESTS = '.print_r(QUICK_CACHE_GET_REQUESTS, TRUE)."\n\n";

exit;

Now load the following into your browser and report the output that you receive.

http://yoursite.com/qc-test.php

Now do it again (this time with a query string) and report the output that you receive.

http://yoursite.com/qc-test.php?test=yes&hello=world
@cnxsoft
cnxsoft commented Aug 9, 2014

http://yoursite.com/qc-test.php output:
GET = Array
(
)

REQUEST = Array
(
)

QUERY_STRING =

QUICK_CACHE_GET_REQUESTS = 0

http://yoursite.com/qc-test.php?test=yes&hello=world output:

GET = Array ( [test] => yes [hello] => world )

REQUEST = Array
(
[test] => yes
[hello] => world
)

QUERY_STRING = test=yes&hello=world

QUICK_CACHE_GET_REQUESTS = 0

@jaswrks
Member
jaswrks commented Aug 9, 2014

@cnxsoft Perfect, that clearly shows that PHP is getting the right data from your WordPress installation. In the second test against (?test=yes&hello=world) I can see that PHP correctly reports both $_GET and $_REQUEST; and that your QC configuration is currently configured not to cache these requests.

😕 I have to say, this is the most confusing issue that I've dealt with in 2014. I just don't see how this is happening yet. If you have a staging server (or anything that I can log into and run tests via FTP and WordPress) that might help. If that's possible, please submit those privately via this contact form: http://www.websharks-inc.com/contact/

What I'm looking for is a place where I can see this occurring, and where I will have FTP access, and access to the WP Dashboard. Preferably on a test WP install running the same configuration where you are experiencing the problem.

@raamdev
Member
raamdev commented Aug 10, 2014

@jaswsinc I'm referencing this WordPress Support thread where another user was experiencing some similarly strange behavior with regards to query strings. I remembered that thread because I recall it being a potential Nginx configuration issue. According to @cnxsoft's original WordPress Support post, he is running "nginx + php-fpm".

Note this comment from Tim Reeves where he shares the part of his Nginx configuration that he believes is causing the strange behavior:

# Catchall: Catch permalinks and special slugs
location ~ ^/.+$ {
 try_files $uri $uri/ /index.php?q=$uri&$args;
}

@cnxsoft Would it be possible for you to share your Nginx configuration with us (excluding any potentially identifying information)?

@raamdev
Member
raamdev commented Aug 10, 2014

@cnxsoft Would it be possible for you to share your Nginx configuration with us (excluding any potentially identifying information)?

Ignore that request, please. I just realized that I already requested (and received) that configuration from you! :) @jaswsinc see https://websharks.zendesk.com/agent/#/tickets/3423

@cnxsoft
cnxsoft commented Aug 10, 2014

Yes, I believe this only happens with nginx, not apache. So this needs to be tested with nginx as I mentioned above. The only way I could provide a testbed is by creating and sharing a VirtualBox image, but this would take a while due to my upload speed.

@jaswrks
Member
jaswrks commented Aug 10, 2014

@raamdev Roger that. I am curious to see what this investigation turns up. I'm not an nginx guru by any means. However, I just don't see (regardless of server configuration) how a PHP script can do the exact opposite of what it's designed to. We verified that PHP is getting what it expects :-)

@jaswrks
Member
jaswrks commented Aug 10, 2014

I am preparing an Nginx install with the configuration file we received. I will report back once I've had a chance to run a full set of tests. I can't wait to see what this turns up. haha

@Expediten

Hi

I had same problem. But i put the setting to: Yes, I would like to cache URLs that contain a query string.
Then all pages in my multisite got cached.

Wordpress Version 3.9.2
quick-cache-pro-v140725
Nginx

@raamdev
Member
raamdev commented Aug 18, 2014

@jaswsinc Did you get a chance to test this with your Nginx install?

@jaswrks
Member
jaswrks commented Aug 18, 2014

No, not yet. On my list though. Probably later today or tomorrow.

@cnxsoft
cnxsoft commented Aug 19, 2014

OK, so I played around with my nginx configuration file.
and changing:
location / {
# This is cool because no php is touched for static content.
try_files $uri $uri/ /index.php;
}
with (as mentioned in http://wiki.nginx.org/WordPress):
location / {
# This is cool because no php is touched for static content.
# include the "?$args" part so non-default permalinks doesn't break when using query string
try_files $uri $uri/ /index.php?$args;
}

Fixed the issue. So now I don't have any duplicate, but somehow there's no debug info at the bottom of the source, despite "Yes, enable notes in the source code.." being selected. I'm not logged in in wordpress when I try, so I'm not sure caching is working. (cached files are being created, but I'm not sure if they are served to visitors).

example URL: http://www.cnx-software.com/2014/08/19/probox2-ex-review/

@jaswrks
Member
jaswrks commented Aug 19, 2014

Thank you! This is on my list for today if at all possible.

@jaswrks
Member
jaswrks commented Aug 19, 2014

Please ignore (for now). I'm just recording the steps that I am taking to reproduce this issue. Once I have completed these tests I will report back again.

Starting with a default Nginx Installation

2014-08-19_05-24-42

@jaswrks
Member
jaswrks commented Aug 19, 2014
@jaswrks
Member
jaswrks commented Aug 19, 2014

Just wanted to NOTE that some of the rules in the Nginx config file submitted in this ticket (https://websharks.zendesk.com/agent/#/tickets/3423, submitted by @cnxsoft), appear to be remnants from a previous W3 Total Cache installation. See the same config here: http://codex.wordpress.org/Nginx#W3_Total_Cache_Rules

@jaswrks
Member
jaswrks commented Aug 19, 2014

I can confirm the following line is the problematic config block for Nginx.

try_files $uri $uri/ /index.php;

This needs to be...

try_files $uri $uri/ /index.php?q=$uri&$args;

Why? I'm not clear on this yet. I will report back once I understand this better. However, I was able to reproduce the issue under the following conditions while running Nginx.

  • Must have friendly permalinks enabled in WordPress to reproduce.
  • Must have the following config block in Nginx, which goes against the WP recommendation.
try_files $uri $uri/ /index.php; # This is invalid according to WordPress.

Should be...

try_files $uri $uri/ /index.php?q=$uri&$args;
@jaswrks
Member
jaswrks commented Aug 19, 2014

It's worth noting that when I reproduced this issue using try_files $uri $uri/ /index.php; and ran a dump of $_SERVER, the REQUEST_URI shows a query string (i.e. /hello-world/?hello=world). However, unlike the tests we ran previously with the help of @cnxsoft, my $_REQUEST and $_GET vars were empty as one would expect them to be; given the issue being reported here.

A dump of $_GET and $_REQUEST whenever I reproduced the issue with
try_files $uri $uri/ /index.php;

Array
(
)
Array
(
)

The REQUEST_URI containing a query string, explains how Quick Cache is able to identify the location and create the .q directory to deal with the query string; even though $_GET is empty in this case. Therefore, QC is behaving as expected here, it is a server misconfiguration that is the underlying cause of this. I am still researching the use of ?q=$uri&$args to determine what this actually is doing behind-the-scenes; i.e. why this is absolutely necessary for WP installs.


UPDATE: the reason for this discrepancy between my tests and the test we ran previously with @cnxsoft was due to the path of the file that we tested previously. In order to trigger this issue, one must be using an internal rewrite. The test we ran previously with @cnxsoft was with a real file; i.e. qc-test.php

@jaswrks
Member
jaswrks commented Aug 19, 2014

Just noting that in WordPress there is a flag that detects the Nginx server.

$GLOBALS['is_nginx']
@jaswrks
Member
jaswrks commented Aug 19, 2014

Just wanted to follow-up here to explain what I learned about $args in the Nginx try_files statement. This does exactly what it appears to do. It passes the query string arguments along through the fastcgi request, ultimately reaching PHP5-FPM. So these are indeed required if you are rewriting URLs with WordPress, and also passing query string variables. Otherwise your server is really not configured to handle everything that it needs to deal with in WordPress.

I experimented quite a bit with various configurations. I found this article provides the most up-to-date and official information. See: http://wiki.nginx.org/WordPress

A Quick Summary...

You need the following configuration blocks in your Nginx config file. Be sure to replace MYSITE.com with your domain name. Additional details here: http://bit.ly/1BybVIf

server {
        listen 80 default_server;
        listen [::]:80 default_server ipv6only=on;

        root /var/www/html;
        index index.php index.html index.htm;

        server_name MYSITE.com;

        location / {
                # try_files $uri $uri/ =404;
                try_files $uri $uri/ /index.php?q=$uri&$args;

                # Or, `try_files $uri $uri/ /index.php?$args;` seems to do fine also.
                # Or, `try_files $uri $uri/ /index.php$is_args$args;` seems to do fine also.
        }

        error_page 404 /404.html;

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
                root /usr/share/nginx/html;
        }

        location ~ \.php$ {
                try_files $uri =404;
                fastcgi_split_path_info ^(.+\.php)(/.+)$;
                fastcgi_pass unix:/var/run/php5-fpm.sock;
                fastcgi_index index.php;
                include fastcgi_params;
        }
}

You should also edit your /etc/php5/fpm/php.ini file and set cgi.fix_pathinfo=0 as suggested by this article at DigitalOcean. See: http://bit.ly/1BybVIf


WARNING: The above configuration file is mostly intended to show the use of try_files. The other portions of this configuration are also necessary, but some of the server paths might be different on your machine. It's best to seek assistance from your hosting company.

@raamdev
Member
raamdev commented Aug 19, 2014

@jaswsinc Thank you VERY MUCH for all your research into this. It's great that we understand this better. I'm going to work on putting together a section on our wiki that contains tips for Quick Cache users who are using Nginx.

It looks like this issue can be closed, as it seems the problem here was a server-side configuration that deviated from the WordPress and Nginx-recommended configuration for a WordPress site, correct?

@jaswrks
Member
jaswrks commented Aug 19, 2014

Fixed the issue. So now I don't have any duplicate, but somehow there's no debug info at the bottom of the source, despite "Yes, enable notes in the source code.." being selected. I'm not logged in in wordpress when I try, so I'm not sure caching is working. (cached files are being created, but I'm not sure if they are served to visitors).

example URL: http://www.cnx-software.com/2014/08/19/probox2-ex-review/

I'd start looking for any plugin that you have installed which might be compressing the output, or be configured to strip comments from the output source code. Also, if there are any Nginx config blocks that might be stripping comments, that could be an issue. If all else fails, try putting together a quick test WP install and see if you can reproduce it there; i.e. a WP install with only QC active, no other plugins.

@jaswrks
Member
jaswrks commented Aug 19, 2014

It looks like this issue can be closed, as it seems the problem here was a server-side configuration that deviated from the WordPress and Nginx-recommended configuration for a WordPress site, correct?

Correct. I think this could be closed now. The only remaining issue is actually a different problem.

@jaswrks
Member
jaswrks commented Aug 19, 2014

Thank you VERY MUCH for all your research into this.

My pleasure. Yep, I learned a thing or two also.

@raamdev raamdev closed this Aug 19, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment