Defeating the browser cache

[jtlapp]

I rolled my own little site generator and discovered a not-so-little problem. When I roll out a new css or js file, clients have to wait for the local cached copy to expire before they get the new version. Even while developing, I find that forcing a reload of the page in Chrome doesn't always reload the files.

It seems that the standard solution is to tack some version indicator to the query portion of the URL. E.g. /site.js?2 or site.js?v=2.

I have also discovered that Twitter caches Twitter card images forever, even if I force a re-validation of the page URL. The recommended solution is also to tack a version to the image URL. E.g. /images/pic.jpg?2.

The simplest solution is to bump the version number of every resource file with every build. But that's crazy. The ideal solution is to only bump the versions of the files that changed. Complicating matters, I construct some of my links programmatically from data files, so the generator wouldn't even know whether a link could be pointing to a changed file until after the link is generated and the linked-to file compared to a prior version. I started imagining a two-phase solution: track links while generating the static files, and then modify already-generated links for files that changed. (And there's also the business of determining whether a file changed relative to the prior upload.)

I'm ready to graduate to a more thought-through site generator and landed on Eleventy as a possibility. This is only one of the big-boy features I'm looking for, but it seems to be the hardest one to find.

How does Eleventy force browsers to reload and ignore cache?

P.S. I do find it necessary to force reload rather than wait for cache expiration. Should HTML, CSS, or JS ever get out of sync, the results can be disastrous.

(Oh, I should mention that it was easy to deal with bumping the image version. I just keep data on my images and append the version to each image URL. Other resources are more problematic because I absolutely don't want to forget to keep them in sync. It's not as crucial that images be kept in sync.)

[jtlapp]

Come to think of it, a build number would work fine, even it makes for strange version numbers in the URLs. But that requires maintaining state between builds. If nonsensical numbers are okay, then the query suffix could be the Unix time or even a random number. Maybe this is simple after all?

[jtlapp]

It looks like I found a solution that doesn't require site generator support. If you do have a standard way to do this in Eleventy, please let me know. Otherwise I'm happy to close this.

[pamtbaau]

It looks like I found a solution that doesn't require site generator support.

Would you mind sharing your solution?

[zachleat]

On my site I use a versioned directory that has far future expires headers set on everything in there. Then when my JS/CSS/etc changes, I bump the version and deploy to the new folder. This allows both the old and new versions to keep working side by side.

e.g.: https://www.zachleat.com/web/dist/1.0.10/defer.min.js

It’s an old trick but it works well https://developer.yahoo.com/performance/rules.html#expires

[zachleat]

Note that Eleventy does not handle this transparently yet, but would support something like this when we deliver #272.

[pamtbaau]

Was just looking at the 'plugin' section from the docs and saw this link: eleventy-plugin-cache-buster

Haven't looked at its code yet, but the description seems to address the 'cache busting' issue.

[pamtbaau]

@zachleat Just curious...

On my site I use a versioned directory that has far future expires headers set on everything in there.

Have you added a .htaccess to the root folder for setting the cache-control on file types?

Then when my JS/CSS/etc changes, I bump the version and deploy to the new folder.

Have you added some package to your css/js workflow to achieve this?

[zachleat]

I think I stole a few apache rules from HTML5 boilerplate to do this yeah:

My website code is up on github if you don’t mind digging through it but I wouldn’t 😅 https://github.com/zachleat/zachleat.com/blob/master/.htaccess#L73

Basically, I use those rules and then create new files under a new versioned url rather than using a cache busting ref. I use the package.json version of my site to decide what the folder is named. So when I bump versions, it bumps the delivered assets for the site. Note that eleventy makes this moderately easier with the pkg global variable: https://www.11ty.io/docs/data/#eleventy-provided-data-variables

[zachleat]

@pamtbaau If you’d like to file a new issue requesting better documentation for this—please do!

[zachleat]

This is an automated message to let you know that a helpful response was posted to your issue and for the health of the repository issue tracker the issue will be closed. This is to help alleviate issues hanging open waiting for a response from the original poster.

If the response works to solve your problem—great! But if you’re still having problems, do not let the issue’s closing deter you if you have additional questions! Post another comment and I will reopen the issue. Thanks!

[jtlapp]

It looks like I found a solution that doesn't require site generator support.

Would you mind sharing your solution?

@pamtbaau, it's just the one I previously described. Set a global variable to either a random number or the Unix time, and then append that number to each resource URL. E.g. site.js?{{unixTime}}.

[jtlapp]

Thank you @zachleat and @pamtbaau for pointing me to other possible solutions. Unfortunately, I'm having to put this project on hold for the moment, but will resume as soon as I can. I've decided to base my rewrite on Eleventy. Thanks so much for the generous assistance!

[dephiros]

Late to the discussion but I just did a quick solution using filter(inspired by hugo asset pipe), can probably extend to calculate hash

  eleventyConfig.addFilter(
    "tsUrl",
    /**
     * @param {string} url
     */ function(url) {
      const [urlPart, paramPart] = url.split("?");
      const params = new URLSearchParams(paramPart || "");
      params.set("ts", `${ts}`);
      console.log(params);
      return `${urlPart}?${params}`;
    }
  );

uses

{{ 'index.js' | url | tsUrl }}

[brycewray]

In the case that the only static assets you need to cache are CSS files (e.g., if you’re letting an external service handle your image assets) and you’re using PostCSS, there’s a PostCSS plugin called PostCSS Hash that makes it ultra-simple. I wrote about it here.

Later edit: Turned out there were flaws with that approach, so please see also this follow-up.