[Proposal] A better way to compile and serve assets

[felixkiss]

I was thinking about a better system to deal with assets generated by asset-pipeline, caching and performance for using it in production.

From all your considerations and decisions you made, it seems to me, that performance in production environments is really important to you.

I was listening to Laravel.io Podcast #3 where Taylor, Eric and Ryan were talking about packages that aim to make your life easier while dealing with assets.

Many people, like Taylor himself, stick with using tools like grunt, CodeKit or guard.

These tools are easy to set up, because they just compile the assets to vanilla js/css sitting in your public directory. You can leave the worry about serving them to your webserver. There will be no cost booting up Laravel just to get correct Routing going.

I can see downsides using one of the above methods in

• losing the flexibility of manifest files
• losing the ability to write your own filters
• the need to set up and install yet another dependencies on every developer's machine
• having to check the compiled files into your SCM, or install the dependencies on the server as well. This means creating noise in your git log by committing the css file, whenever you change the tiniest partial file.

So how about doing it a little different:

In Development

http://foo.dev/assets/some.css still gets served by your Controller. If no app/assets/stylesheets/some.css exists, it will look for app/assets/stylesheets/some.css.scss or app/assets/stylesheets/some.css.less, compile it down to regular css and serve the output.

In Production

For production, the assets get precompiled. Running a php artisan assets:precompile command would be part of the deployment process. It will save your compiled css/js files into public/assets/. Your webserver will happily serve them "as is" – no bootstrapping Laravel for assets anymore in production (where it really counts)!

That way, I think, there is no potential performance or scaling issue caused by this package (no more than by any other file generated by other tools/methods and served as static files by the webserver), while preserving the flexibility of manifest files, custom filters, and more.

Also, you won't have to install node or ruby on your servers and can keep your git history clean.

Lastly, you won't have to keep getting a headache about crazy caching methods. Your code affects fairly non-critical parts of the software development process (development, deployment), that can spare some milliseconds longer execution time.

Thoughts?

[kdocki]

@felixkiss I like it. One question I have though is what files would php artisan assets::precompile put into public/assets/ directory? I mean, would I put every single file over there (like a cache version of it?). I've also got this cache busting issue in too... #35

[evantishuk]

@kdocki, regarding cache-busting and this proposal:

Why couldn't the public/assets/ directory house the static files as suggested and the javascript_include_tag() and stylesheet_link_tag() function just append the SHA'd checksum to the filenames? Something like: /assets/some.css?v=b99fab1320

[felixkiss]

@kdocki I would say it depends on your config file, which files php artisan assets::precompile puts into your public/assets/ directory.

If you have turned concat on, it would only generate the manifest files, otherwise, every single file will get generated.

I tried to look at how sprockets handles cache busting. As far as I understand, it generates some.b99fab1320.css in production and keeps a manifest.json file around that can link some.css => some.b99fab1320.css to generate the appropriate tags via stylesheet_link_tag().

I don't know if this is the ideal approach for this project, though. ?v=b99fab1320 is also a possibility.

All of this only applies to production environments.

[kdocki]

I think a refactor might be in order here... but I haven't decided if this is worth it. The main reason I'm considering this is because if we have like 100's of assets... then reloading the page when doing development is slow. So having the files actually cached to the public/assets directory might make it load seconds quicker which to me might be worth it.

If I do a refactor then I'm going to probably piggy back off assetic more since I know more about it this go around than I did when I first started this thing.

Here's the idea to increase performance (haven't tested yet),

if nginx or apache sees a file in /public/assets/somefile.n32nvn2opq.js then it just serves it like a new file. If that file doesn't exist then the laravel route for /assets/ would kick in and send it to the asset pipeline controller which would then create the file in /public/assets/ and spit it out to the browser. Next time that file is served, it would be there in /public/assets so laravel would never be started.

However, the only reason I'm doing this is to increase performance during development (not production because you can just cache and it loads quick). So is it really worth it?

I've been doing concat => true, cache=>null on my config for the projects I do at work... and it works great except that I loose the line numbers on the files I'm working with since everything is in one giant file. But that hasn't been a problem so far.

[d13r]

I think it would be great to have a unique ID for production assets. Then we could have a very large client-side cache time and the browser wouldn't need to check the Etag, so there would be no need for lots of requests with 304 responses on each page load.

(Also I had a user complaining that my app wasn't working right today, and it was because they had an old stylesheet cached. I thought the Etags and Cache-Control headers should prevent that, but somehow they weren't. Unique IDs in the URL would definitely prevent it though.)

I also agree pre-compiling and having the web server serve the assets directly would be a good idea for production - not that I've had any performance issues myself, but it sounds logical and more scaleable. I wouldn't mind whether they're done on the first request or with a precompile command.

But in development I don't understand what the benefit would be... If the file exists and Apache serves it, then it can't be recompiled when it changes and you'd have to delete the compiled file(s) manually each time you make a change. Or maybe I misunderstood that part?

[kdocki]

@felixkiss @evantishuk hey guys... the refactor is done and out there with the new 4.1.

In this I'm using the assetic cache interface to handle caching (much like evantishuk) suggested. It is technically possible now to create a cache interface driver that writes out files to public/assets or something.

It is also possible to write your own controller action and completely customize how tags are written out when using the and helper functions.

All this to say, asset pipeline should now technically be configurable enough to handle the proposal above with SHA. This means we could ship with a different config file to allow for the kind of behavior asked for in this proposal.

[kdocki]

@davejamesmiller you can completely control the cache now, also I took out the 304 stuff in the refactor because I thought it was making things more difficult for everyone.

[kdocki]

@felixkiss it should now be possible to configure asset pipeline to do exactly what you asked for. Are you still around lol...

[kdocki]

@felixkiss @evantishuk Please re-open if you guys have questions on this proposal.

[felixkiss]

@kdocki yes, I have the repo on watch, so I see everything that is going on here :)

Just not have the time right now to experiment with this, since my bachelor's thesis is due in a few weeks and work is occupying the rest of my time.

I will come back when everything is going back to normal, I guess.

Looking forward to test the new 4.1 version! Until then happy holidays and a good start to 2014 🎉