ExpiresDefault browser cache issue

[webtrainingwheels]

Some customers have complained that they had to clear their browser cache to see new content.

The issue seems to happen when the cache is served from process.php, after it's been cached in the browser.

What's happening is that this rule takes effect:
ExpiresDefault "access plus 1 month"

Instead of the normal HTML expires of 0.

A solution could be to modify our ExpiresDefault to access plus 0 seconds if that doesn't have some other negative impact.
This has worked in cases where it's been applied on customer sites.
Or maybe set some different headers directly in process.php?

For reference, this is how it looks when served from htaccess:

Sample ticket: https://secure.helpscout.net/conversation/508500671/58042?folderId=377611

When testing, remember to uncheck "disable cache" in dev tools!

[arunbasillal]

Related ticket: https://secure.helpscout.net/conversation/509339863/58151?folderId=1724284

@Tabrisrp Since we have explicit rules for all the file types that we know of (and we can possibly add more explicit rules as we discover more file types), would it be a good idea to get rid of the default rule from the .htaccess?

# Perhaps better to whitelist expires rules? Perhaps.
ExpiresDefault                          "access plus 1 month"

Another related ticket: https://secure.helpscout.net/conversation/496865206/56682/?folderId=273766

[arunbasillal]

@webtrainingwheels Hey Lucy, the hacky fix packaged into a helper plugin - https://www.dropbox.com/s/cjv2sridevnqxcq/wp-rocket-helper-htaccess-override-expires-headers.zip?dl=0

Until we have a more permanent and elegant solution.

Here is the diff: https://www.diffchecker.com/jh2Ui0fP

[arunbasillal]

Unconfirmed, but possibly related:

• https://secure.helpscout.net/conversation/536368514/61809?folderId=1724284
• https://secure.helpscout.net/conversation/538106593/62042/?folderId=1724284
• https://secure.helpscout.net/conversation/562780800/65571?folderId=1724284

Related and confirmed:

• https://secure.helpscout.net/conversation/618987835/73698?folderId=1724284

[arunbasillal]

Adding an explanation as to what MIGHT be happening here (on the surface).

Response headers on initial load (or hard refresh) have Content-Type: text/html and our expires header ExpiresByType text/html "access plus 0 seconds" is valid as you can see here:

On a simple refresh, Content-Type is missing and our default expires header ExpiresDefault "access plus 1 month" i.e. 259,2000 seconds is used as you can see here:

While the response is still 304, how the browser handles it MAY vary from browser to browser apparently (this could explain why it shows up on certain mobile browsers). Client-controlled Behavior - https://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.1.6

A client's request MAY specify the maximum age it is willing to accept of an unvalidated response; specifying a value of zero forces the cache(s) to revalidate all responses. A client MAY also specify the minimum time remaining before a response expires. Both of these options increase constraints on the behavior of caches, and so cannot further relax the cache's approximation of semantic transparency.

A Solution?

The helper plugin from here removes the ExpiresDefault and everything works as expected.

[remyperona]

While the solution seems to work, we have no idea of the impact of it on a larger scale.

We also have no idea of the actual root cause of the issue, as for a big majority of the users the current configuration works correctly.

Before implementing it in the plugin, we need to understand why and in what cases it is happening, and make sure it doesn't impact the users for which it works currently.

[webtrainingwheels]

Agree that research is needed.

As for the root cause, I believe it's related to when we added last-modified headers in process.php. BTW I'm not suggesting we remove those because I think they're beneficial, but for me that's the starting point for debugging.

[arunbasillal]

Did some more reading on this matter and it turns out that it is required that a 304 response to not have a content-type.

Here is the response we send:

wp-rocket/inc/front/process.php

Lines 345 to 350 in 16c3c20

	// Checking if the client is validating his cache and if it is current.
	if ( $http_if_modified_since && ( strtotime( $http_if_modified_since ) === @filemtime( $rocket_cache_filepath_gzip ) ) ) {
	// Client's cache is current, so we just respond '304 Not Modified'.
	header( $_SERVER['SERVER_PROTOCOL'] . ' 304 Not Modified', true, 304 );
	exit;
	}

As per this doc: https://tools.ietf.org/html/rfc7232#section-4.1

The server generating a 304 response MUST generate any of the following header fields that would have been sent in a 200 (OK) response to the same request: Cache-Control, Content-Location, Date, ETag, Expires, and Vary.

Doesn't it mean that we should be sending a Cache-Control header? I think that would solve the issue.

Related discussion:

• https://wp-media.slack.com/archives/C43T1AYMQ/p1534495861000100
• https://wp-media.slack.com/archives/C43T1AYMQ/p1534499202000100
• https://youtu.be/uQdQWwRgrJU
• https://www.electrictoolbox.com/php-caching-headers/

[WP-Romain]

New case: https://secure.helpscout.net/conversation/640302005/76753?folderId=1050085

[vmanthos]

Another related ticket:
https://secure.helpscout.net/conversation/653567573/78612/

[vmanthos]

Another ticket here:
https://secure.helpscout.net/conversation/660815262/79757?folderId=2135277