This repository has been archived by the owner on Jan 19, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'master' of github.com:mozilla/anodejsholidayseason
- Loading branch information
Showing
3 changed files
with
462 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,103 @@ | ||
# Fantastic front-end performance, part 2: caching dynamic content with etagify | ||
|
||
You might know that [Connect](https://github.com/senchalabs/connect) puts [ETags](http://en.wikipedia.org/wiki/HTTP_ETag) on static content, but not dynamic content. Unfortunately, if you dynamically generate i18n versions of static pages, those pages don't get caching headers at all--unless you add a build step to pregenerate all pages in all languages. What a lame chore. | ||
|
||
This article introduces [etagify](https://github.com/lloyd/connect-etagify), a Connect middleware that generates ETags on the fly by md5-ing outgoing response bodies, storing the hashes in memory. Etagify lets you skip the build step, improves performance more than you might think (we measured a 9% load time improvement in our tests), and it's super easy to use: | ||
|
||
### 1. register etagify at startup | ||
myapp = require('express').createServer(); | ||
myapp.use(require('etagify')()); // <--- like this. | ||
|
||
### 2. call etagify on routes you want to cache | ||
app.get('/about', function(req, res) { | ||
res.etagify(); // <--- like that. | ||
var body = ejs.render(template, options); | ||
res.send(body); | ||
}); | ||
|
||
Read on to learn more about etagify: how it works, when to use it, when not to use it, and how to measure your results. | ||
|
||
(Need a refresher on ETags and HTTP caching? We've put together a [cheat sheet](https://gist.github.com/6a68/4971859) to get you back up to speed.) | ||
|
||
## How etagify works | ||
|
||
By focusing on a single, concrete use case, etagify gets the job done in just a hundred lines of code (including documentation). Let's take a look at the fifteen lines that cover the basics, leaving out Vary header handling edge cases. | ||
|
||
There are two parts to consider: hashing outgoing responses & caching the hashes; checking the cache against incoming conditional GETs. | ||
|
||
First, here's where we add to the cache. Comments inline. | ||
|
||
// simplified etagify.js internals | ||
|
||
// start with an empty cache | ||
// example entry: | ||
// '/about': { md5: 'fa88257b77...' } | ||
var etags = {}; | ||
|
||
var _end = res.end; | ||
res.end = function(body) { | ||
var hash = crypto.createHash('md5'); | ||
|
||
// if the response has a body, hash it | ||
if (body) { hash.update(body); } | ||
|
||
// then add the item to the cache | ||
etags[req.path] = { md5: hash.digest('hex') }; | ||
|
||
// back to our regularly-scheduled programming | ||
_end.apply(res, arguments); | ||
} | ||
|
||
Next, here's how we check against the cache. Again, comments inline. | ||
|
||
// the etagify middleware | ||
return function(req, res, next) { | ||
var cached = etags[req.path]['md5']; | ||
// always add the ETag if we have it | ||
if (cached) { res.setHeader('ETag', '"' + cached + '"' } | ||
|
||
// if the browser sent a conditional GET, | ||
if (connect.utils.conditionalGET(req)) { | ||
|
||
// check if the If-None-Match and ETags are equal | ||
if (!connect.utils.modified(req, res)) { | ||
|
||
// cache hit! browser's version matches cached version. | ||
// strip out that ETag & bail with a 304 Not Modified. | ||
res.removeHeader('ETag'); | ||
return connect.utils.notModified(res); | ||
} | ||
} | ||
} | ||
|
||
### When (and when not) to use etagify | ||
|
||
Etagify's approach is super simple, and it's a great solution for dynamically-generated pages that don't change while the server is running, like i18n static pages. However, etagify has some gotchas when dealing with other common use cases: | ||
|
||
* if pages change after being first cached, users will always see the stale, cached version | ||
* if pages are personalized for each user, two things could happen: | ||
* if a Vary:cookie header is used to cache users' individual pages separately, then etagify's cache will grow without bound | ||
* if no Vary:cookie header is present, then the first version to enter the cache will be shown to all users | ||
|
||
## Measuring performance improvements | ||
|
||
We didn't foresee huge performance wins with etagify, because conditional GETs still require an HTTP roundtrip, and avoiding page redownloading only saves the user a few KB (see screenshot). However, etagify is a really simple optimization, so even a small gain would justify including it in our stack. | ||
|
||
![firebug screen cap showing 2kb savings](http://i.imgur.com/MVSQYKo.jpg) | ||
|
||
We tested etagify's effects on performance by spinning up a dev instance of [Persona](https://github.com/mozilla/browserid) on an [awsbox](https://github.com/mozilla/awsbox), opening up firebug, and taking 50 load time measurements of our 'about' page--with and without etagify enabled. (Page load times are a good-enough metric for our use case; you might care more about time till above-the-fold content renders, or the first content hits the page, or the first ad is displayed.) | ||
|
||
After gathering raw data, we did some quick statistics to see how much etagify improved performance. We found the mean and standard deviation for both data sets, assuming the measured values were spread out like a [bell curve](http://en.wikipedia.org/wiki/Normal_distribution) around the averages. | ||
|
||
Surprisingly, we found that **etagify reduced load time by 9%**, from 1.65 (SD = 0.19) to 1.50 (SD = 0.13) seconds. That's a serious gain for almost no work. | ||
|
||
Next, we used the [t-test](http://en.wikipedia.org/wiki/Student%27s_t-test) to check the odds that the improvement could be observed without adding etagify at all. Our [p-value](http://en.wikipedia.org/wiki/P-value) was less than 0.01, meaning less than 1% chance that randomness could have caused the apparent improvement. We can conclude that the measured improvement is statistically significant. | ||
|
||
Here's a chart of the averaged before and after data: | ||
|
||
![normal distributions with and without etagify](http://i.imgur.com/Tc45vHg.png) | ||
|
||
## Bringing it all back home | ||
|
||
We think [etagify](https://github.com/lloyd/connect-etagify) is a tiny bundle of win. Even if it's not the right tool for your current project, hopefully our approach of (1) writing focused tools to solve just the problem at hand, and (2) measuring just rigorously enough to be sure you're getting somewhere, gives you inspiration or food for thought. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,187 @@ | ||
# Fantastic front end performance, part 3 - Big performance wins by serving fonts smarter - A Node.js holiday season, part 8 | ||
|
||
We reduced Persona's font footprint 85%, from 300 KB to 45 KB, using font subsetting. This post outlines exactly how we implemented these performance improvements, and gives you tools to do the same. | ||
|
||
## Introducing connect-fonts | ||
|
||
```connect-fonts``` is a Connect font-management middleware that improves ```@font-face``` performance by serving locale-specific subsetted font files, significantly reducing downloaded font size. It also generates locale/browser-specific ```@font-face``` CSS, and manages the CORS header required by Firefox and IE 9+. Subsetted fonts are served from a *font pack*--a directory tree of font subsets, plus a simple JSON config file. Some common open-source fonts are available in pregenerated font packs [on npm](https://npmjs.org/browse/keyword/connect-fonts), and creating your own font packs is straightforward. | ||
|
||
(Feeling lost? We've [put together](https://gist.github.com/6a68/5187976) a few references to good ```@font-face``` resources on the web.) | ||
|
||
### Static vs dynamic font loading | ||
|
||
When you are just serving one big font to all your users, there's not much involved in getting web fonts set up: | ||
* generate ```@font-face``` CSS and insert into your existing CSS | ||
* generate the full family of web fonts from your TTF or OTF file, then put them someplace the web server can access | ||
* add CORS headers to your web server if fonts are served from a separate domain, as Firefox and IE9+ enforce the same origin policy with fonts | ||
|
||
These steps are pretty easy; the awesome [FontSquirrel generator](http://www.fontsquirrel.com/tools/webfont-generator) can generate all the missing font files and the ```@font-face``` CSS declaration for you. You've still got to sit down with Nginx or Apache docs to figure out how to add the CORS header, but that's not too tough. | ||
|
||
If you want to take advantage of font subsetting to hugely improve performance, things become more complex. You'll have font files for each supported locale, and will need to dynamically modify the ```@font-face``` CSS declaration to point at the right URL. CORS management is still needed. This is the problem ```connect-fonts``` solves. | ||
|
||
### Font subsetting: overview | ||
|
||
By default, font files contain lots of characters: the Latin character set familiar to English speakers; accents and accented characters added to the Latin charset for languages like French and German; additional alphabets like Cyrillic or Greek. Some fonts also contain lots of funny symbols, particularly if they support Unicode ([☃](http://en.wikipedia.org/wiki/Snowman#Unicode) anyone?). Some fonts additionally support East Asian languages. Font files contain all of this so that they can ably serve as many audiences as possible. All this flexibility leads to large file sizes; Microsoft Arial Unicode, which has characters for every language and symbol in Unicode 2.1, weighs in at an unbelievable 22 megabytes. | ||
|
||
In contrast, a typical web page only needs a font to do one specific job: display the page's content, usually in just one language, and usually without exotic symbols. By reducing the served font file to just the subset we need, we can shave off a ton of page weight. | ||
|
||
### Performance gains from font subsetting | ||
|
||
Let's compare the size of the localized font files vs the full file size for some common fonts and a few locales. Even if you just serve an English-language site, you can shave off a ton of bytes by serving an English subset. | ||
|
||
Smaller fonts mean faster load time and a shorter wait for styled text to appear on screen. This is particularly important if you want to use ```@font-face``` on mobile; if your users happen to be on a 2G network, saving 50KB can speed up load time by 2-3 seconds. Another consideration: mobile caches are small, and subsetted fonts have a far better chance of staying in cache. | ||
|
||
#### Open Sans regular, size of full font (default) and several subsets (KB): | ||
|
||
![Chart comparing file sizes of Open Sans subsets. Full font, 104 KB. Cyrillic, 59 KB. Latin, 29 KB. German, 22 KB. English, 20 KB. French, 24 KB.](https://gist.github.com/6a68/5122048/raw/23d41ed71f079adb21656518f85e262a0fccaade/unzipped.png) | ||
|
||
#### Same fonts, gzipped (KB): | ||
|
||
![Chart comparing file sizes of Open Sans subsets when gzipped. Full font, 63 KB. Cyrillic, 36 KB. Latin, 19 KB. German, 14 KB. English, 13 KB. French, 15 KB.](https://gist.github.com/6a68/5122048/raw/3f5d6cce8fe1db2721583350dbd01224dd1feb30/gzipped.png) | ||
|
||
Even after gzipping, you can reduce font size 80% by using the English subset of Open Sans (13 KB), instead of the full font (63 KB). Consider that this is the reduction for just one font file--most sites use several. The potential is huge! | ||
|
||
**Using ```connect-fonts```, Mozilla Persona's font footprint went from 300 KB to 45 KB, an 85% reduction.** This equates to several seconds of download time on a typical 3G connection, and up to 10 seconds on a typical 2G connection. | ||
|
||
### Going further with optimizations | ||
|
||
If you're looking to tweak every last byte and HTTP request, ```connect-fonts``` can be configured to return generated CSS as a string instead of a separate file. Going even further, ```connect-fonts```, by default, serves up the smallest possible @font-face declaration, omitting declarations for filetypes not accepted by a given browser. | ||
|
||
## Example: adding connect-fonts to an app | ||
|
||
Suppose you've got a super simple express app that serves up the current time: | ||
|
||
// app.js | ||
const | ||
ejs = require('ejs'), | ||
express = require('express'), | ||
fs = require('fs'); | ||
|
||
var app = express.createServer(), | ||
tpl = fs.readFileSync(__dirname, '/tpl.ejs', 'utf8'); | ||
|
||
app.get('/time', function(req, res) { | ||
var output = ejs.render(tpl, { | ||
currentTime: new Date() | ||
}); | ||
res.send(output); | ||
}); | ||
|
||
app.listen(8765, '127.0.0.1'); | ||
|
||
with a super simple template: | ||
|
||
// tpl.ejs | ||
<!doctype html> | ||
<p>the time is <%= currentTime %>. | ||
|
||
Let's walk through the process of adding ```connect-fonts``` to serve the Open Sans font, one of [several](https://npmjs.org/browse/keyword/connect-fonts) ready-made font packs. | ||
|
||
### App changes | ||
|
||
1. Install via npm: | ||
|
||
$ npm install connect-fonts | ||
$ npm install connect-fonts-opensans | ||
|
||
2. Require the middleware: | ||
|
||
// app.js - updated to use connect-fonts | ||
const | ||
ejs = require('ejs'), | ||
express = require('express'), | ||
fs = require('fs'), | ||
// add requires: | ||
connect_fonts = require('connect-fonts'), | ||
opensans = require('connect-fonts-opensans'); | ||
|
||
var app = express.createServer(), | ||
tpl = fs.readFileSync(__dirname, '/tpl.ejs', 'utf8'); | ||
|
||
3. Initialize the middleware: | ||
|
||
// app.js continued | ||
// add this app.use call: | ||
app.use(connect_fonts.setup({ | ||
fonts: [opensans], | ||
allow_origin: 'http://localhost:8765' | ||
}) | ||
The arguments to ```connect_fonts.setup()``` include: | ||
* ```fonts```: an array of fonts to enable, | ||
* ```allow_origin```: the origin for which we serve fonts; ```connect-fonts``` uses this info to set the Access-Control-Allow-Origin header for browsers that need it (Firefox 3.5+, IE 9+) | ||
* ```ua``` (optional): a parameter listing the user-agents to which we'll serve fonts. By default, ```connect-fonts``` uses UA sniffing to only serve browsers font formats they can parse, reducing CSS size. ```ua: 'all'``` overrides this to serve all fonts to all browsers. | ||
|
||
4. Inside your route, pass the user's locale to the template: | ||
|
||
// app.js continued | ||
app.get('/time', function(req, res) { | ||
var output = ejs.render(tpl, { | ||
// pass the user's locale to the template | ||
userLocale: detectLocale(req), | ||
currentTime: new Date() | ||
}); | ||
res.send(output); | ||
}); | ||
5. Detect the user's preferred language. Mozilla Persona uses [i18n-abide](https://github.com/mozilla/i18n-abide), and [locale](https://github.com/jed/locale) is another swell option; both are available via npm. For the sake of keeping this example short, we'll just grab the first two chars from the [Accept-Language header](https://developer.mozilla.org/en-US/docs/HTTP/Content_negotiation#The_Accept-Language.3A_header): | ||
|
||
// oversimplified locale detection | ||
function detectLocale(req) { | ||
return req.headers['accept-language'].slice(0,2); | ||
} | ||
|
||
app.listen(8765, '127.0.0.1'); | ||
// end of app.js | ||
|
||
### Template changes | ||
|
||
Now we need to update the template. ```connect-fonts``` assumes routes are of the form | ||
|
||
/:locale/:font-list/fonts.css | ||
for example, | ||
|
||
/fr/opensans-regular,opensans-italics/fonts.css | ||
In our case, we'll need to: | ||
|
||
5. add a stylesheet ```<link>``` to the template matching the route expected by ```connect-fonts```: | ||
|
||
// tpl.ejs - updated to use connect-fonts | ||
<!doctype html> | ||
<link href="/<%= userLocale %>/opensans-regular/fonts.css" rel="stylesheet"> | ||
|
||
6. Update the page style to use the new font, and we're done! | ||
|
||
// tpl.ejs continued | ||
<style> | ||
body { font-family: "Open Sans", "sans-serif"; } | ||
</style> | ||
<p>the time is <%= currentTime %>. | ||
|
||
The CSS generated by ```connect-fonts``` is based on the user's locale and browser. Here's an example for the 'en' localized subset of Open Sans: | ||
|
||
// this is the output with the middleware's ua param set to 'all'. | ||
@font-face { | ||
font-family: 'Open Sans'; | ||
font-style: normal; | ||
font-weight: 400; | ||
src: url('/fonts/en/opensans-regular.eot'); | ||
src: local('Open Sans'), | ||
local('OpenSans'), | ||
url('/fonts/en/opensans-regular.eot#') format('embedded-opentype'), | ||
url('/fonts/en/opensans-regular.woff') format('woff'), | ||
url('/fonts/en/opensans-regular.ttf') format('truetype'), | ||
url('/fonts/en/opensans-regular.svg#Open Sans') format('svg'); | ||
} | ||
|
||
If you don't want to incur the cost of an extra HTTP request, you can use the ```connect_fonts.generate_css()``` method to return this CSS as a string, then insert it into your existing CSS files as part of a build/minification process. | ||
|
||
That does it! Our little app is serving up stylish timestamps. This example code is available on [github](https://github.com/6a68/connect-fonts-example) and [npm](https://npmjs.org/package/connect-fonts-example) if you want to play with it. | ||
|
||
We've covered a pre-made font pack, but it's easy to create your own font packs for paid fonts. There are instructions on the ```connect-fonts``` [readme](https://github.com/shane-tomlinson/connect-fonts). | ||
|
||
## Wrapping up | ||
|
||
Font subsetting can bring huge performance gains to sites that use web fonts; ```connect-fonts``` handles a lot of the complexity if you self-host fonts in an internationalized Connect app. If your site isn't yet internationalized, you can still use ```connect-fonts``` to serve up your native subset, and it'll still generate ```@font-face``` CSS and any necessary CORS headers for you, plus you'll have a smooth upgrade path to internationalize later. | ||
|
||
### Future directions | ||
|
||
Today, ```connect-fonts``` handles subsetting based on locale. What if it also stripped out font hinting for platforms that don't need it (everything other than Windows)? What if it also optionally gzipped fonts and added far-future caching headers? There's cool work yet to be done. If you'd like to contribute ideas or code, we'd love the help! Grab the [source](https://github.com/shane-tomlinson/connect-fonts) and dive in. |
Oops, something went wrong.