multiple threads to compile assets?

[josh-m-sharpe]

When compiling assets, I generally only see one processor being used, which tells me the whole asset compilation process is single-threaded. Is there a reason why it must be that way?

I imagine using threads could significantly improve compiling times.

[ghost]

Any news so far? I would like this possibility too, since compilation may consume alot of time and this way, we could greatly reduce deployment time.

[jeremy]

Nobody's working on it, @robertwachs. Please do investigate!

[steel]

I have a fork that handles multi-threaded manifest.compile()
It's located at https://github.com/steel/sprockets/tree/multi-process-compiler

On an 8 core box running with 8 manifest compiler workers we've dropped asset compilation from 7 minutes (1 core intel box) down to 50 seconds. Not too shabby.

I haven't had time to cleanup the code and issue a pull request yet but it does work.

[jjb]

@steel awesome!

[ghost]

Nice solution!
Is there any way using this with Rails 3.2.2 in a clean way? I tried to bundle it but sadly, ActionPack depends on sprockets ~> 2.1.2 so it wont use this version. Also tried to backport it but the structure of Sprockets changed alot.

If someone has a solution for this, please leave a note. Thanks

Edit Seems like @guilleiguaran took care of that: rails/rails#5409 but its not in 3.2 stable yet.

[steel]

Boy its been a long time but finally dropped the monkey patch into a gem:

https://rubygems.org/gems/sprockets-derailleur

You can now compile your manifest with as many processes as you need and I've included a processor detector as well.

processes = SprocketsDerailleur::number_of_processors rescue 1
manifest = Sprockets::Manifest.new(Application::Sprockets, 'public/assets', processes)

We use this multiple times a day deploying all over the place. Hope this helps other out there.

[chamnap]

@steel awesome!