multiple threads to compile assets? #308

Closed
josh-m-sharpe opened this Issue Mar 19, 2012 · 7 comments

Projects

None yet

6 participants

@josh-m-sharpe

When compiling assets, I generally only see one processor being used, which tells me the whole asset compilation process is single-threaded. Is there a reason why it must be that way?

I imagine using threads could significantly improve compiling times.

@ghost
ghost commented Apr 21, 2012

Any news so far? I would like this possibility too, since compilation may consume alot of time and this way, we could greatly reduce deployment time.

@jeremy
jeremy commented Apr 21, 2012

Nobody's working on it, @robertwachs. Please do investigate!

@steel
Contributor
steel commented Apr 28, 2012

I have a fork that handles multi-threaded manifest.compile()
It's located at https://github.com/steel/sprockets/tree/multi-process-compiler

On an 8 core box running with 8 manifest compiler workers we've dropped asset compilation from 7 minutes (1 core intel box) down to 50 seconds. Not too shabby.

I haven't had time to cleanup the code and issue a pull request yet but it does work.

@josh josh closed this May 16, 2012
@jjb
jjb commented Jun 3, 2012

@steel awesome!

@ghost
ghost commented Jun 4, 2012

Nice solution!
Is there any way using this with Rails 3.2.2 in a clean way? I tried to bundle it but sadly, ActionPack depends on sprockets ~> 2.1.2 so it wont use this version. Also tried to backport it but the structure of Sprockets changed alot.

If someone has a solution for this, please leave a note. Thanks

Edit Seems like @guilleiguaran took care of that: rails/rails#5409 but its not in 3.2 stable yet.

@steel
Contributor
steel commented Jan 11, 2013

Boy its been a long time but finally dropped the monkey patch into a gem:

https://rubygems.org/gems/sprockets-derailleur

You can now compile your manifest with as many processes as you need and I've included a processor detector as well.

processes = SprocketsDerailleur::number_of_processors rescue 1
manifest = Sprockets::Manifest.new(Application::Sprockets, 'public/assets', processes)

We use this multiple times a day deploying all over the place. Hope this helps other out there.

@chamnap
chamnap commented Nov 24, 2014

@steel awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment