Cache bust #1350

ghost · 2017-04-22T18:50:36Z

Template middleware return a jade compile function
minified or not with unique hash by template, the client side
get a template hash from a list where live availables templates,
aditional set a max-age expiration for better render performance
in client-side and is more friendly with development because not
need build templates manually.

Closes: #1204

strugee

Hey, so sorry - this dropped off my radar somehow and I forgot about it :(

I started going through and leaving comments but stopped since I'm not sure this is the best approach - AFAICT this compiles templates each startup and caches them in memory, right? I'm not sure this is better than having a build step that writes templates back to disk, similar to what we currently have. That makes startup faster and it improves memory usage.

I'm not really sure, honestly.

strugee · 2017-08-19T02:31:28Z

lib/template.js

+//
+// Template middleware with cache-buster
+//
+// Copyright 2012, E14N https://e14n.com/


This should be adjusted :)

strugee · 2017-08-19T02:32:04Z

lib/template.js

+var fs = require("fs"),
+    path = require("path"),
+    crypto = require("crypto"),
+    _ = require("underscore"),


Since this was submitted we've migrated to Lodash so this needs to be adjusted. Sorry, that's my bad :(

strugee · 2017-08-19T02:33:07Z

lib/template.js

+        "// \n" +
+        "// @licend  The above is the entire license notice\n" +
+        "// for the JavaScript code in this page.\n\n" +
+        "// XXX: this needs to be broken up into 3-4 smaller modules \n";


This probably doesn't belong here?

strugee · 2017-08-19T02:33:29Z

lib/template.js

+        "// @licstart  The following is the entire license notice for the \n" +
+        "//  JavaScript code in this page.\n" +
+        "// \n" +
+        "// Copyright 2011-2013, E14N https://e14n.com/\n" +


This should probably read:

Copyright 2011-2017, E14N https://e14n.com/ and contributors

strugee · 2017-08-19T02:34:23Z

lib/template.js

+        "// XXX: this needs to be broken up into 3-4 smaller modules \n";
+
+
+    // Get al files from views directory but only take all .jade files


all instead of al

strugee · 2017-08-19T04:11:20Z

lib/template.js

+                    templateContent = fs.readFileSync(templatePath, {encoding: "utf8"}),
+                    // Compile jade template for client side
+                    templateFn = jade.compileClient(templateContent, {
+                        externalRuntime: false,


What is this option? I don't see it in the docs; maybe you meant inlineRuntimeFunctions?

In jade 1.11.x both optionsexternalRuntime or inlineRuntimeFunctions not exist, see: https://github.com/pugjs/pug/tree/master/packages/pug-runtime pugjs/pug@1eeb9e6

ghost · 2017-09-02T00:26:49Z

Memory is always faster than read from disk (example swap vs zswap), reduce IO-Operations is always better for performance, this option is also more friendly for frontend developers and don't increment much the startup time only increment seconds more

https://www.backblaze.com/blog/whats-diff-ram-vs-storage/
https://stackoverflow.com/questions/25467993/node-js-performance-reading-files-once-or-on-every-request

coveralls · 2017-10-09T16:13:43Z

Changes Unknown when pulling 0cdf6ec on vxcamiloxv:cache-bust into ** on pump-io:master**.

coveralls · 2017-10-09T16:44:58Z

Coverage increased (+0.2%) to 73.974% when pulling 572780a on vxcamiloxv:cache-bust into 8fb1962 on pump-io:master.

coveralls · 2017-10-09T17:12:29Z

Coverage increased (+0.2%) to 73.974% when pulling 572780a on vxcamiloxv:cache-bust into 8fb1962 on pump-io:master.

coveralls · 2017-10-16T05:47:01Z

Coverage decreased (-3.6%) to 70.187% when pulling c7c3b8b on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T05:47:02Z

Coverage decreased (-3.6%) to 70.187% when pulling c7c3b8b on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T05:47:02Z

Coverage decreased (-3.6%) to 70.187% when pulling c7c3b8b on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T06:30:48Z

Coverage decreased (-3.7%) to 70.149% when pulling 9c7412f on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T07:13:57Z

Coverage increased (+0.2%) to 73.96% when pulling 9c7412f on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T07:13:57Z

Coverage increased (+0.2%) to 73.96% when pulling 9c7412f on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T07:13:57Z

Coverage increased (+0.2%) to 73.96% when pulling 9c7412f on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T18:28:39Z

Coverage increased (+0.2%) to 73.96% when pulling 7f76f89 on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T18:28:39Z

Coverage increased (+0.2%) to 73.96% when pulling 7f76f89 on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-16T18:28:39Z

Coverage increased (+0.2%) to 73.96% when pulling 7f76f89 on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-17T18:01:42Z

Coverage increased (+0.2%) to 73.96% when pulling d07e89e on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

This middleware returns a jade template compile function minified or not with unique hash by template, the client side get a template hash from a list where live availables templates, aditional set a max-age expiration for better render performance in client-side and is more friendly with development because not need build templates manually. Related: #1204

'inlineRuntimeFunctions' option not exist in jade 1.x, Added for compatibility in future 'pug' upgrades

Close: #1462

coveralls · 2017-10-17T19:30:26Z

Coverage increased (+0.2%) to 73.975% when pulling 59ee061 on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

coveralls · 2017-10-17T19:46:39Z

Coverage increased (+0.2%) to 73.975% when pulling 59ee061 on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

ghost · 2017-10-18T02:50:06Z

@strugee I was thinking about other ways to make better this implementation and if you still thinking keep in memory could be have performance issues I have two options.

Server side options

A) The template (should be rename to cachebuster) middleware currently read a template from disk and serve this to client in debug mode, I could be will keep this behavior in production but with pre-compile and minified templates (and other statics)

B) I'll modified the request path and remove the hash, Will delegate the response to express-statics middleware.

Client side options.

A) Reads all statics files and make hash, keeps in memory (only name file and hash)

For template, a templates.js with files names and hash for every file or append to Pump._templates in layout.jade as JSON
For other statics reads all from disk and generate hash, keeps in memory.

B) A build task (a script in package.json) for generate a JS file and requires as node dependency, the file could be have this structure:

exports {
    js: [
        {"file-name": "hash"}
    ],
    css: [
        {"file-name": "hash"}
    ],
    template: [
        {"file-name": "hash"}
    ]
}

For templates same as option A
For other statics generate the file url, like this:

script(src=getFileHash("/javascript/dist/pump.min.js"))

and the results

<script src="/javascript/dist/pump.min.hdg6252.js"></script>

Generation options

A) Generate and minified all statics in for example javascript/dist/ and templates/dist/ by a script task in package.json

B) Generate, minified and saves all statics by server on startup (developer friendly)

What do you think?

coveralls · 2017-10-18T03:24:47Z

Coverage decreased (-3.6%) to 70.187% when pulling c7c3b8b on vxcamiloxv:cache-bust into d1037f2 on pump-io:master.

ghost · 2017-10-24T14:14:35Z

I close this PR because has 9 months, and I was thinking in a better and more generic approach for that (not only templates), I Will work over options the previous comment.

ghost mentioned this pull request Apr 25, 2017

Uglify template JS #1357

Open

ghost mentioned this pull request Jul 3, 2017

Compiled Jade doesn't comply with LibreJS #1366

Open

strugee reviewed Aug 19, 2017

View reviewed changes

Camilo QS added 12 commits October 17, 2017 13:23

Get templates with hash for cache-buster

07c4604

Get realtime template in debug mode (not from memory)

5cfc84a

Get template without hash as fallback

b261df2

Add unit tests for template middleware

e235f2d

Fix typo

6a67a6e

Add uglify for minify templates

720964d

Chnage to strict validation statement

aa5733b

Change underscore dependency to lodash

07f54fd

Update copyright notice

5a3f647

Fix typo

0729d6e

Update jade/pug compile option

c32f00c

'inlineRuntimeFunctions' option not exist in jade 1.x, Added for compatibility in future 'pug' upgrades

Camilo QS added 14 commits October 17, 2017 13:39

Update unit test dependencies

e37d26b

Update to uglify 3.x

f918523

Change copyright notice

6e36e3e

Refactor template middleware

e585663

Disable templates compile script

36e599b

Add logs for template middleware

614ed16

Add docs for template middleware

c75b629

Remove some verbose log

fbb56cb

Use httputil HEAD instead of zombie in some cases

6ec8605

Fix multiple requests for same template

49170f3

Close: #1462

Remove template hash validation because is unnecessary

98d2689

Update unit test for templates without hash

24c0099

Fix connection refuse in unit test for templates

81ff3b4

Change copyright notice

59ee061

ghost closed this Oct 18, 2017

ghost reopened this Oct 18, 2017

ghost closed this Oct 24, 2017

ghost mentioned this pull request Mar 15, 2018

Bundle pump.js with Browserify #1539

Open

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cache bust #1350

Cache bust #1350

ghost commented Apr 22, 2017 •

edited by ghost

strugee left a comment

strugee Aug 19, 2017

ghost Sep 1, 2017

strugee Aug 19, 2017

ghost Sep 1, 2017

strugee Aug 19, 2017

strugee Aug 19, 2017

ghost Sep 1, 2017

strugee Aug 19, 2017

ghost Sep 1, 2017

strugee Aug 19, 2017

ghost Sep 1, 2017

ghost commented Sep 2, 2017 •

edited by ghost

coveralls commented Oct 9, 2017 •

edited

coveralls commented Oct 9, 2017 •

edited

coveralls commented Oct 9, 2017 •

edited

coveralls commented Oct 16, 2017 •

edited

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017 •

edited

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017 •

edited

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017 •

edited

coveralls commented Oct 17, 2017 •

edited

coveralls commented Oct 17, 2017 •

edited

coveralls commented Oct 17, 2017 •

edited

ghost commented Oct 18, 2017 •

edited by ghost

coveralls commented Oct 18, 2017 •

edited

ghost commented Oct 24, 2017

		"// XXX: this needs to be broken up into 3-4 smaller modules \n";


		// Get al files from views directory but only take all .jade files

Cache bust #1350

Cache bust #1350

Conversation

ghost commented Apr 22, 2017 • edited by ghost

strugee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Sep 2, 2017 • edited by ghost

coveralls commented Oct 9, 2017 • edited

coveralls commented Oct 9, 2017 • edited

coveralls commented Oct 9, 2017 • edited

coveralls commented Oct 16, 2017 • edited

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017 • edited

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017 • edited

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017

coveralls commented Oct 16, 2017 • edited

coveralls commented Oct 17, 2017 • edited

coveralls commented Oct 17, 2017 • edited

coveralls commented Oct 17, 2017 • edited

ghost commented Oct 18, 2017 • edited by ghost

Server side options

Client side options.

Generation options

coveralls commented Oct 18, 2017 • edited

ghost commented Oct 24, 2017

ghost commented Apr 22, 2017 •

edited by ghost

ghost commented Sep 2, 2017 •

edited by ghost

coveralls commented Oct 9, 2017 •

edited

coveralls commented Oct 9, 2017 •

edited

coveralls commented Oct 9, 2017 •

edited

coveralls commented Oct 16, 2017 •

edited

coveralls commented Oct 16, 2017 •

edited

coveralls commented Oct 16, 2017 •

edited

coveralls commented Oct 16, 2017 •

edited

coveralls commented Oct 17, 2017 •

edited

coveralls commented Oct 17, 2017 •

edited

coveralls commented Oct 17, 2017 •

edited

ghost commented Oct 18, 2017 •

edited by ghost

coveralls commented Oct 18, 2017 •

edited