Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Fingerprinting: MD5 digest vs modified time #16

Closed
andrewdeandrade opened this Issue Aug 23, 2012 · 8 comments

Comments

Projects
None yet
3 participants
Contributor

andrewdeandrade commented Aug 23, 2012

Currently, express-cdn uses modified time as its cache invalidation strategy. Rails 2.x used to use this approach and switched to an MD5 digest approach for reasons outlined here:
http://edgeguides.rubyonrails.org/asset_pipeline.html#what-is-fingerprinting-and-why-should-i-care

Since I am using less stylesheets, I ran into problems with the mtime approach because every the css file would be recompiled with a new mtime even though the file contents remain identical to previous compiles.

PS Google has a good document describing caching strategies here:
https://developers.google.com/speed/docs/best-practices/caching

Owner

niftylettuce commented Aug 23, 2012

Do you suggest using Git's integrated hash?

Contributor

andrewdeandrade commented Aug 23, 2012

The approach I used on my fork is simply to perform something like this:

var hash = crypto.createHash("md5");

// ...
for (var b=0; b<assets.length; b+=1) {
  // ...
  hash = hash.update(fs.readFileSync(path.join(options.publicDir, assets[b])));
  // ...
}
// ...
return createTag(src, "/" + name, attributes) + "\n";

The problem with this approach at the moment is that it is computationally intensive and time consuming to generate an MD5 digest of a bunch of files every time you want to generate an HTML tag. I'm guessing that the best approach is to generate the MD5 digest only once and cache it in memory.

Here is my commit with this naive implementation:
andrewdeandrade/express-cdn@3e2744a

I'm going to contact bminer and check if he'd be interested in modularizing his node-static-assets module to separate the express middleware from the cache strategies.

Tell me more about what you had in mind regarding the git integrated hash approach. I'm not sure I understand. Do you have a link to an implementation?

Contributor

andrewdeandrade commented Sep 11, 2012

Nick, here's a naïve implementation.

https://github.com/malandrew/express-cdn

It's not production ready as I need to go ahead and modify the middleware part so it caches values and doesn't recalculate on every server request. Any ideas on how you'd go about implementing that caching?

Once I figure out how best to implement this, I want to go ahead and add this to the less-middleware module since it's also got the mtime achille's heel that is the source of edgecase bugs.

Owner

niftylettuce commented Sep 20, 2012

Andrew,

Sorry for delay -- been working hard on my startup -- if you do a pull
request I have no problem looking it over!

Someone integrated CSS for url(...) attribute in the newest version of
express-cdn v0.0.6 -- check it out.

It needs the appended mtime to the image though, might you look into that?
That was one of the lazyweb requests.

On Tue, Sep 11, 2012 at 12:44 AM, Andrew de Andrade <
notifications@github.com> wrote:

Nick, here's a naïve implementation.

https://github.com/malandrew/express-cdn

It's not production ready as I need to go ahead and modify the middleware
part so it caches values and doesn't recalculate on every server request.
Any ideas on how you'd go about implementing that caching?

Once I figure out how best to implement this, I want to go ahead and add
this to the less-middleware module since it's also got the mtime achille's
heel that is the source of edgecase bugs.


Reply to this email directly or view it on GitHubhttps://github.com/niftylettuce/express-cdn/issues/16#issuecomment-8446953.

Andrew, couldn't you set once:true on the middleware to only have it generate the compiled css once? I believe the problem is that express-cdn goes straight to disk to find the css files without requesting it through the middleware, no?

Owner

niftylettuce commented Sep 29, 2012

@grydstedt for CSS we do a GET request https://github.com/niftylettuce/express-cdn/blob/master/lib/main.js#L279 -- if that answers your question? /cc @malandrew

Owner

niftylettuce commented Oct 18, 2012

we should take care of this issue, timestamps aren't too cool right now. @grydstedt did you have any ?'s about how CSS gets requested or anything?

Owner

niftylettuce commented Apr 16, 2013

adding to lazyweb requests in the readme, I think git sha might be better/easier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment