Skip to content

Detect asset changes by saving source_digests in manifest.yml, and speed up non-digest assets #21

Closed
wants to merge 2 commits into from
@ndbroadbent

This feature speeds up the asset pipeline by only recompiling assets whose dependencies have changed. It does this by changing manifest.yml to include :source_digests as well as the digests for compiled assets under the :digest_files key.

This pull request depends on two others: sprockets #367 and rails #7767 (guides update)

Source Digests

By computing source digests, we can quickly determine if the asset needs to be recompiled or not. If nothing has changed in the sources of an asset's dependencies, then compilation is skipped for that asset. Source digests are calculated by removing all processors except ERB, and disabling compression, so the contents of source files are concatenated and hashed. Please see pull request #367 on Sprockets for the required changes.

Generating Non-digest Assets

Instead of compiling twice for both digest and non-digest assets, non-digest assets are now generated by stripping the digests out of the already compiled css and js files. This means we don't have to process and compress the same files twice. I'm making the assumption that asset paths are the only differences between digest/non-digest assets, but please let me know if you can see any problems with this approach.

Rails 3.2.x Support

I've prepared patches for Rails 3.2.8 and Sprockets 2.1.3 on branches named rails3_assets_speedup. I've extracted these patches into a gem called turbo-sprockets-rails3, so that myself and others can start using the feature.
Do you think there's any chance of this feature making it into the Rails 3.2.9 release?

Tests

All the tests are passing for both sprockets and sprockets-rails. I've added one or two tests, but there already seemed to be decent coverage for the areas I was changing. Please let me know if more tests are needed.

Testing this pull request

You'll need to add the following lines to the Gemfile of your Rails 4 app:

gem 'sprockets',       github: 'ndbroadbent/sprockets',       branch: 'allow_assets_without_processing'
gem 'sprockets-rails', github: 'ndbroadbent/sprockets-rails', branch: 'dont_recompile_unchanged_assets'

You should also add the following lines to config/environments/production.rb to view the logs for rake assets:precompile:

config.log_level = :debug
config.logger = Logger.new(STDOUT)

Before / After

Here's some asset compilation times for a smallish Rails app.
(running time RAILS_ENV=production RAILS_GROUPS=assets rake assets:precompile)

Before:

asset:precompile: 26.993s

After:

Task Time
Initial asset:precompile 18.525s
assets:precompile:primary 9.772s
assets:precompile:nondigest 64ms
Unchanged assets 9.386s
assets:precompile:primary 877ms
assets:precompile:nondigest 91ms
One changed application.js dependency 12.386s
assets:precompile:primary 3.910s
assets:precompile:nondigest 48ms
@ndbroadbent ndbroadbent referenced this pull request in capistrano/capistrano Sep 27, 2012
Closed

built-in rails asset compile: skip if no changes to assets? #227

@radar
Ruby on Rails member
radar commented Sep 28, 2012

I :heart: you.

@ndbroadbent

Thanks! :D

@jrochkind jrochkind referenced this pull request in capistrano/capistrano Sep 28, 2012
Closed

Per-server conditional asset precompilation #174

@schneems
Ruby on Rails member

:+1: :heart:

@route
route commented Oct 2, 2012

Also, instead of compiling twice for both digest and non-digest assets, non-digest assets are now generated from digest assets, with digests stripped from the content of any css and js files. This means we don't have to process and compress the same files twice. I'm making the assumption that asset paths are the only differences between digest/non-digest assets, but please let me know if you can see any problems with this approach.

Yea, it seems good if it compiles non-digest assets just for 64ms, WDYT @guilleiguaran @josevalim ? Relating comment #8 (comment)

@ndbroadbent ndbroadbent Detect asset changes by adding source_digests to manifest.yml
This allows us to quickly determine if the asset needs to be
recompiled. If nothing has changed in the source assets,
then compilation is skipped.
Also, instead of compiling twice for non-digest assets,
digest assets are copied instead, and the known digests are
stripped from the body of css and js files.
The copied js/css files are also gzipped.
746d1ff
@jeremy
Ruby on Rails member
jeremy commented Oct 7, 2012

Made some comments on the backport to Rails 3-2-stable @ rails/rails#7866

@zoras
zoras commented Oct 15, 2012

:+1:

@josh josh closed this Oct 18, 2012
@asymmetric

Hey @josh, was this closed because it got merged, or for some other reason?

@guilleiguaran
Ruby on Rails member

@asymmetric was closed since it doesn't apply anymore with the new version of sprockets-rails

@Simbul
Simbul commented Dec 5, 2012

Sorry to reanimate a closed thread.
@guilleiguaran, when you say "doesn't apply anymore", does it mean it's already in the new version of sprockets-rails? Or is it incompatible?

I guess what I really want to know is: what's the status of the effort towards speeding up the pipeline? Is that being tracked anywhere? Is turbo-sprockets-rails3 the answer to that?

@schneems
Ruby on Rails member
schneems commented Dec 5, 2012

I'm interested to know as well. We're in the vetting process of getting this behavior fully supported on the Heroku buildpack with the gem for now (heroku/heroku-buildpack-ruby#44), but will want Rails 4 asset caching to work out of the box preferably.

In my very un-professional benchmarks, this behavior dropped precompiling assets on the codetriage.com app from 20 seconds to 5 seconds. One of our biggest consistent complaints is deploy time, and asset compilation has been a huge bottleneck.

@williscool

I'm interested in whether or not this was integrated as well

@brauliobo

Was caching acomplished? I see some references above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.