• Optimizing asset bundling and serving with Rails

    kneath 19 Nov 2009

    We spend a lot of time optimizing the front end experience at GitHub. With that said, our asset (css, javascript, images) packaging and serving has evolved to be the best setup I've seen out of any web application I've worked on in my life.

    Originally, I was going to package what we have up into a plugin, but realized that much of our asset packaging is specific our particular app architecture and choice of deployment strategy. If you haven't read up on our deployment recipe read it now. I cannot stress enough how awesome it is to have 14 second no downtime deploys. In any case, you can find the relevant asset bundling code in this gist

    Benefits of our asset bundling

    • Users never have to wait while the server generates bundle caches, ever. With default Rails bundling, each time you deploy, each request until your server generates the bundle has to wait for the bundle to finish. This makes your site pause for about 30s after each deploy.
    • We can use slower asset minifiers (such as YUI or Google Closure) without consequence to our users.
    • Adding new stylesheets or javascripts is as easy as creating the file. No need to worry about including a new file in every layout file.
    • Because we base our ASSET_ID off our git modified date, we can deploy code updates without forcing users to lose their css/js cache.
    • We take full advantage of image caching with SSL while eliminating the unauthenticated mixed content warnings some browsers throw.

    Our asset bundling is comprised of several different pieces:

    1. A particular css & js file structure
    2. Rails helpers to include css & js bundles in production and the corresponding files in development.
    3. A rake task to bundle and minify css & javascript as well as the accompanying changes to deploy.rb to make it happen on deploy
    4. Tweaks to our Rails environment to use smart ASSET_ID and asset servers

    CSS & JS file layout

    Our file layout for CSS & JS is detailed in the README for Javascript, but roughly resembles something like this:

    public/javascripts
    |-- README.md
    |-- admin
    | |-- date.js
    | `-- datePicker.js
    |-- common
    | |-- application.js
    | |-- jquery.facebox.js
    | `-- jquery.relatize_date.js
    |-- dev
    | |-- jquery-1.3.2.js
    | `-- jquery-ui-1.5.3.js
    |-- gist
    | `-- application.js
    |-- github
    | |-- _plugins
    | | |-- jquery.autocomplete.js
    | | `-- jquery.truncate.js
    | |-- application.js
    | |-- blob.js
    | |-- commit.js
    `-- rogue
        |-- farbtastic.js
        |-- iui.js
        `-- s3_upload.js
    

    I like this layout because:

    • It allows me to namespace specific files to specific layouts (gist, github.com, iPhone, admin-only layouts, etc) and share files between apps (common).
    • I can lay out files however I want within each of these namespaces, and reorganize them at will.

    Some might say that relying on including everything is bad practice -- but remember that web-based javascript is almost exclusively onDOMReady or later. That means that there is no dependency order problems. If you run into dependency order issues, you're writing javascript wrong.

    Rails Helpers

    To help with this new bundle strategy, I've created some Rails helpers to replace your standard stylesheet_link_tag and javascript_include_tag. Because of the way we bundle files, it was necessary to use custom helpers. As an added benefit, these helpers are much more robust than the standard Rails helpers.

    Here's the code:

    Our application.html.erb now looks something like this:

    <%= javascript_dev ['jquery-1.3.2', "#{http_protocol}://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"] %>
    <%= javascript_bundle 'common', 'github' %>
    

    This includes jQuery and all javascript files under public/javascripts/common and public/javascripts/github (recursively). Super simple and we probably won't need to change this for a very long time. We just add files to the relevant directories and they get included magically.

    For pages that have heavy javascript load, you can still use the regular javascript_include_tag to include these files (we keep them under the public/javascripts/rogue directory).

    Bundle rake & deploy tasks

    The javascript_bundle and stylesheet_bundle helpers both assume that in production mode, there'll be a corresponding bundle file. Since we are proactively generating these files, you need to create these manually on each deploy.

    Throw this into lib/tasks/bundle.rake and the corresponding YUI & Closure jars and then run rake bundle:all to generate your javascript. You can customize this to use the minifying package of your choice.

    To make sure this gets run on deploy, you can add this to your deploy.rb:

    Tweaks to production.rb

    The last step in optimizing your asset bundling for deploys is to tweak your production.rb config file to make asset serving a bit smarter. The relevant bits in our file are:

    There's three important things going on here.

    First— If you hit a page using SSL, we serve all assets through SSL. If you're on Safari, we send all CSS & images non-ssl since Safari doesn't have a mixed content warning.

    It is of note that many people suggest serving CSS & images non-ssl to Firefox. This was good practice when Firefox 2.0 was standard, but now that Firefox 3.0 is standard (and obeys cache-control:public as it should) there is no need for this hack. Firefox does have a mixed content warning (albeit not as prominent as IE), so I choose to use SSL.

    Second— We're serving assets out of 4 different servers. This fakes browsers into downloading things faster and is generally good practice.

    Third— We're hitting the git repo on the server (note our deployment setup) and getting a sha of the last changes to the public/stylesheets and public/javascripts directory. We use that sha as the ASSET_ID (the bit that gets tacked on after css/js files as ?sha-here).

    This means that if we deploy a change that only affects app/application.rb we don't interrupt our user's cache of the javascripts and stylesheets.

    Conclusion

    What all of this adds up to is that our deploys have almost no frontend consequence unless they intend to (changing css/js). This is huge for a site that does dozens of deploys a day. All browser caches remain the same and there isn't any downtime while we bundle up assets. It also means we're not afraid to deploy changes that may only affect one line of code and some minor feature.

    All of this is not to say there isn't room for improvement in our stack. I'm still tracking down some SSL bugs, and always trying to cut down on the total CSS, javascript and image load we deliver on every page.

  • Comments

    jashkenas Thu Nov 19 13:33:53 -0800 2009

    If you'd like to get a lot of these optimizations for free in your Rails project, check out the Jammit gem. This is a great post, Kyle. Thanks for going into such comprehensive detail.

    veged Thu Nov 19 15:02:23 -0800 2009

    try use <%= javascript_dev ['jquery-1.3.2', "//ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"] %>
    instead of <%= javascript_dev ['jquery-1.3.2', "#{http_protocol}://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"] %>

    chriseppstein Thu Nov 19 15:17:28 -0800 2009

    FYI: if you guys used compass and sass for your stylesheets, the images referenced in your stylesheets would be timestamp invalidated and asset-host'd just like they are using rails helpers (just use image_url instead of url)

    kneath Thu Nov 19 15:22:00 -0800 2009

    @chris: Given the way CSS works, the asset hosting stuff happens no matter if you use sass or not. Timestamped images to me aren't preferred since it would force users to re-download all the site's images (including layout images) every time we deploy. Images referenced in our CSS almost never change, and I'm happy to manually change them if we do change them.

    loe Thu Nov 19 17:48:12 -0800 2009

    @kyle

    Are you saying that using url('/image/foo.jpg') (not run through image_path which would generate the query string and serve it off of an asset hostname) in a stylesheet that is served with a far future expires is ok? Will the image be re-downloaded if the asset id on the stylesheet is changed or will the browser still use the cached image set to expire in 30 years?

    I've been wanting to run all stylesheets through ERB before deploys to get the asset hosting benefits, but if it only helps me by using the asset hosts it might not be worth the work.

    kneath Thu Nov 19 18:20:07 -0800 2009

    @loe: Images referenced in stylesheets are relative to the stylesheet's location, not the page being served. So if you serve a CSS file off of asset1.github.com, all images will be downloaded from asset1.github.com. The image's cache and expiry has nothing to do with the CSS, only whatever headers the actual image sends. If you set a far-future expiry for an image, the image should remain cached for a very long time.

    jaknowlden Fri Nov 20 23:14:56 -0800 2009

    Out of curiosity, could you comment on the use of existing open source asset bundling rails plugins that exist out there and why you chose not to use them. Especially those hosted on github ;)

    kneath Sat Nov 21 11:53:36 -0800 2009

    @jaknowlden: The choice was pretty simple — most of the functionality described above has been in the GitHub codebase for a very long time. It would have been a lot of work to move to a new asset bundling plugin without any benefits.

    kastner Sun Nov 22 00:16:37 -0800 2009

    What impact is there while the bundle:all is running? What about if a client requests it during generation?

    kneath Mon Nov 23 16:41:57 -0800 2009

    @kastner: Because of the way our deployment recipe works, the old bundle_* don't get removed (we use git reset on a current directory, not symlinks). So they get replaced with the new ones gracefully, and after all of them have been generated, the unicorns get restarted. So any app/view code depending on new js doesn't get served up until everything's been bundled and already available. Because we restart after bundling, our asset ids don't get changed until the new bundles are on the server too -- so there's no chance of someone caching an old file as a new one.

    mcmire Tue Nov 24 12:36:50 -0800 2009

    What about per-page Javascript or CSS? Do you just put it straight in the view (and therefore it won't be bundled)?

    schof Thu Dec 10 05:26:07 -0800 2009

    Thanks for this fantastic write-up. I was about to attempt a similar custom solution but you've already done all the hard work for me! I have one question about asset hosts in general. Suppose you're going to host your image assets on S3. What would you recommend for this type of setup? How many hosts? Should I limit the total to 4 hosts still? Maybe 2 hosts for JS and CSS and 2 hosts for S3? Just curious what your recommendation would be.

    taf2 Thu Dec 17 17:19:55 -0800 2009

    wonder could you just version the compiled asset bundles? If no assets have changed then the compiled asset would still remain the same... I would imagine for css/js this would be pretty small over head for git to handle...

    wimleers Thu Dec 24 17:49:55 -0800 2009

    Have you thought about JS minification, image optimization (size reduction without quality loss) etc?

    I've built something that automates this as part of my bachelor thesis and it's hosted right here, at GitHub: http://github.com/wimleers/fileconveyor.>

    Please log in to comment.