[Question] - Cache Busting

[dazinator]

ASP.NET 5 has a handy way of cache busting via the asp-append-version tag:

<link rel="stylesheet" href="~/Css/somecssfile.min.css" asp-append-version="true"/>

This appends some file version hash to the url - which will mean the url changes when the file changes:

<link rel="stylesheet" href="/css/somecssfile.min.css?v=UdxKHVNJA5vb1EsG9O9uURFDfEE3j1E3DgwL6NiDGMc" />

How does cache busting work with Smidge? Can we just continue to use the asp-append-version tag helper?

[Shazwazza]

Hi! I was unaware of the asp-append-version tag. Do you know how that gets generated? We need to ensure that it's not changed on each app restart since Smidge caches the processed files on disk to serve so they don't need to be re-processed on app restart. Smidge currently uses the Smidge config: https://github.com/Shazwazza/Smidge/wiki/installation#install to version/cache-bust the files using the 'version' (which can be a string too)

[dazinator]

Ah I see. So we are effectively talking about 2 different levels of cache here. There is the browser caching the resource. Then there is Smidge caching the bundle.

With smidge caching the bundle, am I correct in understanding that Smidge wont update its cache unles syou manually increment the version number in the config?

I'm wondering if you would be open to improving that mechanism, so that perhaps Smidge will auotmaitcally update its cache when the contents of the bundle changes. I have an idea for how that can work and it basically means computing a hash from the files in the bundle when the bundle file is created, and appending that hash to the bundles file name:

12345-[HASH].js

Then, when Smidge is asked to create a bundle, it can compute the hash, compare the hash to the existing cache file's hash for the bundle, if its different, delete the old cache file and create the new bundle file. Obviously we want the hash computation to be much cheaper than creating a bundle, as otherwise its pointless - you may as well create a new bundle file every time!

So this will add an additional bit of processing time to deal with hash computation, but I think it will have advantages over the version flag.

The other level of caching to this, which is cache busting in the browser, is that when Smidges bundle file changes (which would be as per the above, or currenlty when the version number is manually changed) - then we need the browser to request a new resource, so it doesn;t use its cache. Currently for smidge, if the bundle file name is changing when the new bundle file is generated, then we shouldn't need to do anything. i.e we should see this from version to version:

v1:

<link rel="stylesheet" href="~/App_Data/12345.css"/>

v2:

<link rel="stylesheet" href="~/App_Data/67891.css"/>

If however the bundle filename is staying the same for new versions of the bundle, then we would need to append some query string, such as the new "version" so that the browser would see it as a new resource, even though the file name is staying the same:

v1

<link rel="stylesheet" href="~/App_Data/12345.css?Version=1"/>

v2

<link rel="stylesheet" href="~/App_Data/12345.css?Version=2"/>

So I think we only really need to worry about the client side cache busting if the bundle file names generated by smidge for new versions of the bunlde are staying the same?

[Shazwazza]

It would be great to be able to compute the hash for pre-defined bundles and that should be relatively 'easy', the bigger issue with auto-versioning is when pre-defined bundles are not used and inline/non-named declarations are used. When this happens, there is no hash that can be determined up-front and needs to be determined when the URL is being generated, but since this can dynamically change and there's no name associated with this, there's really no possible way to track a previous hash and therefore the Smidge version would need to be used (hope that makes sense)

Smidge automatically appends the version to the processed bundles so you don't have to do that manually, however it doesn't do this for debugging assets (this could easily be an option though).

[dazinator]

is when pre-defined bundles are not used and inline/non-named declarations are used.

By that, are you referring to this API:

        SmidgeHelper.RequiresJs("~/Js/Folder*js");
        SmidgeHelper.RequiresCss("~/Css/Folder*css");

When this happens, there is no hash that can be determined up-front and needs to be determined when the URL is being generated, but since this can dynamically change and there's no name associated with this, there's really no possible way to track a previous hash

I think I understand what you mean. Perhaps we could infer a bundle name based on the contents of the bundle in that scenario - of the top of my head, how about, combine the individual file names of the files in the bundle into a long string, then then compute a hash of that string, and use that hash as the bundle name? I don't think that is a perfect solution, as when people add new files to the bundle, we will no longer be able to tie that bundle back up to the its previous version, so this may result in an ever growing cache directory :) But atleast, it would provide a way for us to auto update the cache, when the files in the bundle remain the same, but are being edited. The file would essentially consist of 2 hashes:

[hash for bundle name]-[hash for version comparison].css

[Shazwazza]

That might be able to work (the load balance implications are a bit different though), I'm also not so worried about a growing cache directory

[dazinator]

Just had a further thought around this. Rather than computing a hash based on the contents of the bundle, could potentially just use the "last modifed" date time of the most recently modified file in the bundle. This would be much quicker to check. As soon as a file in the bundle was edited, this should no longer match, and so coukd be the signal to invalidate the cache

[IDisposable]

I wouldn't want the cache to be based on the file date/time given modern source control systems idiotic idea to checkout files with the current date/time

[dazinator]

Ah good point. I suspected it was too ideal to be a good idea!

[Shazwazza]

For the next minor version I'll make the versioning/cache busting part an abstraction so at least the logic to generate the version can be replaced by anything. This abstraction will need to take into account the versioning for pre-defined bundles, as well as the inline declarations

That will at least be a good start :)

[dazinator]

Just adding this link here in case its interesting. This is what MVC uses to create a version which is appenced to urls:

https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNetCore.Mvc.TagHelpers/Internal/FileVersionProvider.cs

[Shazwazza]

Going to close this now since this is available in 2.0: #51

[John-Blair]

Could you advise? My smidge bundle urls always have v1 appended to them - see below - even when I have configured a different version number in appsettings.json

| <link href="/sb/JBMasterCSS.css.v1" rel="stylesheet">   | <link href="/sb/JBPageSectionsCSS.css.v1" rel="stylesheet">
My config in appsettings.json is currently set to the following
"smidge": { "dataFolder": "Smidge", "version": "22" }

and I've verified it gets read in at runtime - so I was expecting my current version 22 to be part of the url e.g. /sb/JBMasterCSS.css.v22 - a bit like ClientDependency used to generate.
How do I get the url version to change?

Also I cannot find any dataFolder called "Smidge" where is it located or is it virtual?
Thanks.

 

[John-Blair]

FYI I found that if I change the Version number in the "RuntimeMinification" setting then it appears in the url. I find this confusing that there is 2 locations for version now - the other under the "smidge" section in the configuration. Should only be 1 place.

"RuntimeMinification": { "UseInMemoryCache": false, "CacheBuster": "Version", "Version": "33" },

[Shazwazza]

That is Umbraco's implementation and usage of Smidge.