Investigate implementation of r.js as Sprockets compressor

[jwhitley]

On the road to the 0.5.0 release, I've noticed that it may be possible to do a somewhat clean implementation of r.js as a Sprockets compressor. This issue tracks research along those lines, and an implementation if deemed appropriate.

[andrewdeandrade]

John,

We're currently thinking about using this gem of yours. Out of curiosity, do you have an ETA on when r.js support will be included?

(PS to answer your question from the require.js google group: I don't see any reason to support the sprockets directives. Require.js AFAICT allows all the flexibility necessary to include things in the project. TBH, the only thing I don't like about the Rails Asset pipeline is the separation of the javascript across 3 areas of the project (app/assets, lib/assets and vendor/assets). IMHO, all the client-side stuff should drop off one directory in the project.)

[jwhitley]

HI Andrew,

I'm currently working on this FT, expecting the 0.5.0 release within the next week. That will have precompilation support via r.js, with r.js configuration parameters (paths, modules, etc.) passed in via config/requirejs.yml.

The roadmap is to let it bake out for a short period and tackle any really critical (or really easy) issues that arise, then bump it 1.0.

You can investigate the current state of things on the precompilation branch, but it's definitely in an alpha state right now. That branch resolves #1, but has two issues that need to be addressed before I'm ready to release it into the wild:

• The r.js side of rake assets:precompile breaks (due to a Sprockets issue) if a build is already present in public/assets. Workaround: run rake assets:clean beforehand.
• Per Module digests should represent aggregated contents #10, we aren't computing the aggregated digest for the built modules via r.js yet. This issue relates to a possible solution to that problem.

Thanks for the feedback re: Sprockets directives. I've come to the same conclusion as I've gotten further into this project. While one certainly could use Sprockets directives in the current implementation, it's really a lot cleaner to manage dependencies entirely via AMD.

[andrewdeandrade]

Is the config/requirejs.yml file the same as putting require.config({}); in the first file before your first require? (e.g. the file where you define paths, baseUrl, etc). If so, why make it a yml file instead of just maintaining native js form?

Will the build profile (e.g. app.build.js) go in as a file in app/assets/javascripts?

I'm not sure what you mean by Sprockets' aggregated digest. I looked online and in my project and didn't see anything named digest. Is this the app.build.js file or a file built automatically by sprockets for each of the concatenated/uglified/minified script files that go to production?

I found some files with occurrences of the term "hexdigest", but I didn't really understand what purpose they serve. Is it to fingerprint the files to see which ones have changed for caching purposes?

I definitely agree that it is a lot cleaner to manage dependencies entirely via AMD. While I like peanut butter in my chocolate and vice versa. I prefer to keep my ruby out of my javascript and vice versa.

(Excuse me if any of the questions above sound dumb. I'm kind of new to build scripts and deploying.)

[mrwade]

+1 I can't wait for the 0.5.0 release!

[jwhitley]

Implementing r.js as a Sprockets compressor turns out to be neither possible or necessary for the requirejs-rails precompilation. Closing this issue. Current ETA for 0.5.0 release by EOD Thursday; just pending critical docs updates and a bit of final shakeout testing.

[jwhitley]

@malandrew I ended up leaving config/requirejs.yml as-is for now. Will solicit and consider feedback on this before 1.0 release, likely via the requirejs Google group.

[andrewdeandrade]

Cool deal. Why did it turn out not to be necessary or possible? Can't the sprockets recompilation just call r.js directly?

[jwhitley]

HI @malandrew,

The problem (and about 80% of the work for 0.5.0) comes down to the impedance mismatch between Sprockets and r.js:

• Sprockets has APIs that allow processors to express inclusion of assets distinct from using Sprockets directives (e.g. Sprockets::Context#require_asset), and for expressing non-included dependencies (Sprockets::Context#depend_on), mostly intended for files used by the processor that impact build output but which aren't included).
• In spite of those APIs, Sprockets can't delegate the actual composition of asset files to the processor. In essence, Sprockets somewhat expects that its processors work solely on a single-file basis. That defeats the whole point of using r.js to produce an optimized build, either single-file or layered.
• Even if Sprockets could delegate the layered build to r.js, it would be a slow and/or painful to extract the needed dependency information from the assets. To do it sanely basically requires a separate run of r.js. @jrburke has an experimental branch to try this out. I thought that I might need it, but ultimately avoided it altogether.

Long story short, when it comes to producing a suite of built assets, Rails+Sprockets and r.js compete head-to-head. r.js is far and away the preferable solution for folks using require.js, so the gem's policy is to delegate all build responsbilities to r.js. That said, requirejs-rails does integrate with and use some Sprockets features, particularly the asset paths arrangement. Among other things, this means that any gem that provides JavaScript assets (ala jquery-rails) works out-of-the-box with requirejs-rails in development mode as well as in a production deployment using built assets.

[andrewdeandrade]

Actually, the way you described the way it needs to be done is how I expected it to work. Delegating everything to r.js to me is preferable because it is has the greatest separation of concerns and also is the most portable solution.

The only thing I would expect sprockets to do is tell r.js which build file to run (e.g. development, test, production).

I'm honestly not a huge fan of Sprockets' c-preprocessor like approach. Less coupling between the require.js client and sprockets also means that there is room to go possible switch some or all of the backend to node.js in the future and leave rails serving legacy API content. IMHO, the javascript assets are already too intertwined with how rails works already. Only when the client and server are running the same language (e.g. node), does it make sense to have them tightly coupled.

I'll take a look at 0.5.0 when it's out and also jburke's experimental branch. Thanks!