Collapse Key Dependencies via Sprockets?

[wied03]

Not if this has been thought of or is feasible but most of the time, when people are developing with Rails, etc., they may not need each Opal (or for that matter RSpec) file supplied to their browser as an individual file. That creates a lot of requests, which even with caching/304 responses, slows down the browser during development (production is not a problem because everything gets precompiled).

Might it be possible to tweak Sprockets to make this happen? It would be kind of an asset precompile on the fly but only for certain assets.

I know it's easy to programmatically compile using the Sprockets::Manifest class but that's the easy part. The hard part is changing the behavior as assets are required/served. It can't be done in Sprockets::Server because that's too late in the cycle.

[wied03]

I suppose this would need to be done in Opal::Processor under sprockets by overriding the evaluate method and, if the filename includes 'opal.rb', spin up a separate sprockets context that precompiles that file, then returns that.

Is that a good approach? Is this a problem worth solving?

[elia]

That's surely a valuable proposal, especially for speeding up requests in (sprockets) debug mode by cutting down the number of requests.

I see two ways to do this, the first is adding some kind of option to the processor that will render opal as a single file with no dependencies, that would go directly into opal.

The other is to add something like opal-collapsed.js.erb to opal-rails implemented as follows:

//= depend_on opal
<%= Rails.application.assets['opal'].to_s %>

@wied03 @adambeynon any preference?

[wied03]

@elia, I suppose I didn't think about how easy it is (using the erb approach you described) to get sprockets to dump the entire thing to a string.

That said, it seems like both of these approaches would be needed. Otherwise, how would you hook into Sprockets when it's processing all of the app source that includes require 'opal' ? You would still need to do that using some approach that tweaks the Processor, right?

It also might not be just Opal (I could think of doing this for opal-rspec right out of the gate as well).

[elia]

This sounds much alike the framework blackboxing feature in Chrome, we could have a list of assets for which the collapsing is on:

Opal::Config.collapsed_paths = %w[opal opal-rspec]

About the processor approach I'd have to try first but I think the context environment can be used to replicate basically the same thing sprockets does (pseudocode):

if Opal::Config.collapsed_paths.include? logical_path
  dependencies_code = dependencies.map {|d| context.environment[d.logical_path].to_s}
  result = "#{dependencies_code}#{result}"
end

On the other hand the soft .js.erb approach can be extended at will to any asset

[wied03]

@elia - Yeah, Rails.application.assets is a Sprockets::Environment object so that would make sense.

Remaining questions:

1. Should this go in core opal? Seems like the answer is yes since that's where the Sprockets code is
2. OK to go ahead with a pull request for core opal?
3. It could be broken out into 2 settings: collapsed_paths and a enable_collapse. Separate from that, what should the default be? I would imagine that not collapsing by default is more sensible than forcing it.

[elia]

I'd really like to hear @adambeynon thoughts on this, I started experimenting with the erb solution in a project (using exactly the code above) and works great.

The more I think about it the more comes to mind that should be a sprockets (or sprockets-rails) feature instead. I'll open an issue there and see what they think.

In the meantime if you need this kind of feature on some project you can go with the .js.erb solution until this is sorted out.

[wied03]

That would be good. In trying to do this in the meantime (with opal-rspec under opal-rails, I'll do the actual Rails app run later), I've also noticed a lot of the opal requires are unpacked before Opal::Processor is ever called in this scenario but I can't figure out exactly how (I know OpalSpecController creates the high level temp file using Opal::Rails::SpecBuilder but I can't track down what converts that to JS. If I set breakpoints in Opal::Processor.call or Opal::TiltTemplate I never see that temp file come in). The temp file looks fine (example below) I just can't figure out what is exploding the first require 'opal'. Any idea here?

temp file contents (tmp/opal_spec/opal_spec_runner_a884f369a73a02d07b78915eceaed573d85032af.js.rb)

require "opal"
require "opal-rspec"
require "components/base/html_ajax_spec"
require "components/base/option_spec"
require "components/base/select_label_spec"
require "components/base/select_spec"
require "components/form/space_spec"
require "components/mixins/collection_upfront_spec"
require "components/mixins/html_spec"
Opal::RSpec::Runner.autorun

[elia]

Ouch

Yes, that part is quite messy, let me see if I can fix it tonight

Ideally I'd require the spec_helper to "require" Opal-rspec. That way you're free to require it as opal-rspec-collapsed.

[wied03]

@elia I also sent you an email about an opal-rspec PR (opal/opal-rspec#19). I don't think that's relevant here but wondered what the thoughts were on that

[wied03]

@elia If things are changing towards a combined asset anyways, then maybe this problem mainly sticks around on the testing side. One possible option is to do something like Karma does where you keep a 'server' process and browser (phantom) sitting there running and can re-run tests quickly.

I don't know exactly how that works (phantom constantly polling or what not). I guess the Rails server would need to stay up and then the Rake task. I would say maybe Spring can help here but I'm not sure where the delay is exactly.

[elia]

I've been using the collapsed (via erb) version for a while now and the difference is quite noticeable. I'll try to bake the collapsing feature right into the processor so that any lib can be processed that way.

[wied03]

How do you do use that exactly? Do you go and replace all the require 'opal' statements in your code with require 'opal-collapsed'?

[elia]

Yes, but I tipically have just one require for opal