Skip to content

ActionDispatch::Static serves precompressed assets #8669

Closed
wants to merge 2 commits into from

10 participants

@jbaudanza

Sprockets currently generates gzipped javascript and css files next to their uncompressed counterparts. However, to take advantage of this you need to use an nginx extension or something similar.

This patch will allow ActionDispatch::Static to serve up these gzipped files when appropriate.

For example, this request:

curl -H "Accept-Encoding: gzip" http://localhost:3000/assets/application.js

Will look for application.js.gz on disk and respond with:

 HTTP/1.1 200 OK
 Content-Type: application/javascript
 Content-Encoding: gzip
 Content-Length: 1234
 More-Headers...

 <compressed data>

This is particularly helpful for environments like Heroku, that depend on ActionDispatch::Static to serve static assets. Currently, the only way to serve gzipped assets in this environment is to install Rack::Deflate ahead of ActionDispatch::Static. This is undesirable, because Rack::Deflate compresses everything, including compressed assets like images and swfs.

jbaudanza added some commits Dec 31, 2012
@jbaudanza jbaudanza Removed ActionDispatch::FileHandler abstraction.
This class was originally added to support multiple file roots.
Since this feature no longer exists, the extra abstraction is unnecessary.
bdcee3e
@jbaudanza jbaudanza ActionDispatch::Static will serve pre-gziped assets a600cff
@peterkeen

Looks useful. +1

@pixeltrix
Ruby on Rails member

Agreed that the functionality is useful but I'm not sure that we should be building a slow, poorly specified web server as part of Rails - I think Rack::Static should be patched to handle this kind of thing and ActionDispatch::Static be a shim to that to provide backwards compatibility.

@josevalim @jeremy @tenderlove wdyt?

@jbaudanza

Thanks for your feedback @pixeltrix. Your suggestion sounds reasonable, and I'd be happy to resubmit this as a patch to Rack::Static.

But if the Rack guys don't want this, I do think this should be handled by Rails somewhere. Given that we've already gone down the path of generating these gzipped assets via the asset compiler, it seems reasonable that rails should be capable of serving these files up to the client somehow.

@jelder
jelder commented Jan 1, 2013

+1

@jbaudanza jbaudanza referenced this pull request in rack/rack Jan 1, 2013
Closed

Serving gzipped assets from Rack::Static #478

@maletor
maletor commented Feb 21, 2013

Yes +1

@pixeltrix
Ruby on Rails member

@maletor there's an open PR (rack/rack#479) that implements this functionality which seems like it will be eventually accepted once the details are ironed out - probably better to post there if you're interested in it.

@pixeltrix pixeltrix closed this Feb 21, 2013
@jbaudanza

I don't think this is going to make it into Rack. Might we reconsider adding this to Rails?

@eliotsykes

Another option for serving gzipped assets until its available in Rails core: https://gist.github.com/eliotsykes/6049536

@jbaudanza

@pixeltrix Could we reconsider adding this to Rails? There seems to be a demand for it, and there doesn't seem to be any traction getting this into Rack.

@jbaudanza

@guilleiguaran Could we consider reopening this?

I don't think this is making it into Rack, as there hasn't been a response there in 10 months.

This functionality feels more at home in Rails, since the asset pipeline is generating these files. Rack seems more like an interface library.

I'd love to have this rolled out for all the Rails apps on Heroku that are unnecessarily serving uncompressed javascript and css. CC @schneems

This is also much more graceful than using Rack::Deflate on your static assets, which will blindly recompress jpegs and pngs.

@eliotsykes

Both @jbaudanza and I maintain separate gems that offer this functionality. It'd be a kindness to us both to know the Rails core team's direction on whether serving .gz assets is likely to be Rails' responsibility in the near future.

If gz serving is not going to become Rails' responsibility, it may be worth removing the gz creation from Rails entirely.

For developers new to Rails, it's a little disconnected to have the asset pipeline create .gz assets and then not use them. I suspect this will be burning a fair few folk on PaaSs like Heroku (where having a server like nginx handle asset requests is uncommon) and degrading performance for their apps' end-users.

Another option might be to modify the asset precompilation step to give clarification to developers who haven't stumbled upon this intricacy of the asset pipeline and give them a gentle nudge toward improving the performance of their app:

  1. Make gz asset creation a config option that is off by default, or
  2. Add an info message to the console output with something like "Gzip compressed assets have been created. These will not be served by your app unless you: a) configure your web server (e.g. nginx, Apache) to serve them or b) use a 3rd party gem or c) write your own middleware
@pixeltrix
Ruby on Rails member

@eliotsykes as I've said previously I'm loathe to try and build a poorly specified web server using ActionDispatch::Static - we don't recommend it's use in production environments and the only reason it's enabled on Heroku is so that single dyno test applications don't have to go through the hassle of setting up external asset hosts.

As for disabling gz asset creation by default I don't think that forcing everyone that doesn't use Heroku to have to manually specify an extra option is a good choice. I don't think the warning would be much help either as it would just get lost in the noise of cap deploy.

I still think getting it integrated into Rack is the best option - from re-reading the discussion it seems as though the sticking points are implementation details rather than any philosophical objection.

@searls
searls commented Jun 11, 2014

👍

I understand @pixeltrix's reasoning, but seeing as CDNs like Cloudfront require assets to be compressed by the origin server (which, for most Heroku applications, is Heroku) in order to serve gzipped assets on the CDN network, then I think there's a strong argument to be made for at least an opt-in configuration to serve up static assets when they're available.

I agree that nobody wants ActionDispatch::Static to pretend to be a production grade static asset server, but in this particular Heroku + custom origin-backed Cloudfront configuration (and perhaps many others, scant few developers even seriously look into this), behavior like this is actually required to serve gzipped assets to end-users in production at all.

@interlock

👍 I'm with @searls.

Also, let people make their own poor production decisions if need be. In some cases, ActionDispatch::Static is the appropriate production solution for some projects.

@schneems
Ruby on Rails member

If you're running a production site you should be using a CDN. period. ActionDispatch::Static actually slows down Heroku apps, as all requests will stat the disk for EVERY request, this is slow and bad. If you're serving behind a CDN then you only need to serve your assets once per every invalidation.

I would like to see an approach that does not put a slow tax on every request just because you want to serve assets. One approach might be mounting a sprockets related middleware only on paths that match /assets/and then we know it's an asset, and we know it will have a static precompiled file and we can do the right thing.

Pushing this functionality into Rack seemed like a good idea, but if we do that we're adding an even slower tax to those that want to run their app without being tied down by an extra non-ruby webserver. Since we're in Rails we can make a non-general purpose tool that works really well for our needs, serving assets. I have many thoughts about this, but i'm working on putting them into something coherent.

Thanks for chiming in here, i think this is a valid concern that we should be thinking about. I'm still 👎 on this exact implementation, but we should talk about a way to get gzipped assets (and non-gzipped assets) without adding a slow tax to every request.

@searls
@eliotsykes

With zopfli gzip now being a better alternative for compression vs regular gzip in the asset pipeline, I'm wondering out loud here if we should reconsider removing gzip compression from Rails entirely or offer some kind of notice to developers when it's used that they could be giving their users something better... Should Rails be concerned with the finer points of file compression? I'm undecided.

@schneems
Ruby on Rails member
schneems commented Sep 3, 2014

@eliotsykes rails should be concerned with the best practices in web development. If people are leaving a huge performance gain on the table by serving gzip assets over the alternative we should look into supporting that alternative. For this discussion some benchmarks could be good.

This feature works by simply reading a file extension off of disk. That file is provided by sprockets, so...
If sprockets starts supporting another compression method and browsers support receiving and decompressing that algorithm, i'll be happy to add support in the middleware layer provided the benchmarks work out in our favor.

@eliotsykes

Thanks. Sprockets had considered doing something with zopfli over a year ago: sstephenson/sprockets#439 . It looks like that pull request wasn't merged. Pinging @josh and @sstephenson in case they'd like to contribute to this discussion.

I don't have any benchmarks - the oft-quoted number is 5% smaller files vs non-zopfli gzip. For those interested in a little more detail (from http://en.wikipedia.org/wiki/Zopfli):

Zopfli Compression Algorithm is an open sourced, zlib-compatible data compression library developed by Jyrki Alakuijala and Lode Vandevenne. It achieves typically 5% better compression than zlib but takes 81 or 100 times longer to compress the same data. Decompression speed is unaffected. Google says it is "best suited for applications where data is compressed once and sent over a network many times, for example, static content for the web"

The slower speed isn't noticeable in my experience. Even if it was, the asset gzipping is a one-off pre-deployment task during rake assets:precompile.

@schneems
Ruby on Rails member
schneems commented Sep 3, 2014

if the algorithm produces a .gz we'll still serve it, so I think there's nothing rails would need to do. I will say that the assets:precompile is a pain in my side for deployment speed. 67% of the time spent deploying all Rails 4 apps on Heroku is spent in rake assets:precompile 😱

@eliotsykes

Zopfli is 100% gzip compatible. Gzip clients handle it fine, it produces *.gz files.

I agree the assets:precompile time can be frustrating. I don't know what most of that time is spent on (Sass/CoffeeScript compilation? Gzipping?).

When I compare that time to some Jenkins thumb-twiddling builds from my past though it's a drop in the ocean. If zopfli added 5 minutes to asset precompilation (unlikely) I'd still do it; better me paying that price than users having a slower experience due to larger assets.

@josh
Ruby on Rails member
josh commented Sep 3, 2014

I think this feature should be dropped completely. In fact, Sprockets 3.x will not write out .gz by default going forward.

Theres so many issues configuring you server to do this correctly when the gzip file is already written to disk.

See https://issues.apache.org/bugzilla/show_bug.cgi?id=39727

Web server should be configured to do on the fly transfer encoding rather than serving content encoding assets from disk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.