Skip to content
This repository has been archived by the owner before Nov 9, 2022. It is now read-only.

multiple threads to compile assets? #308

Closed
josh-m-sharpe opened this issue Mar 19, 2012 · 7 comments
Closed

multiple threads to compile assets? #308

josh-m-sharpe opened this issue Mar 19, 2012 · 7 comments

Comments

@josh-m-sharpe
Copy link

josh-m-sharpe commented Mar 19, 2012

When compiling assets, I generally only see one processor being used, which tells me the whole asset compilation process is single-threaded. Is there a reason why it must be that way?

I imagine using threads could significantly improve compiling times.

@ghost
Copy link

ghost commented Apr 21, 2012

Any news so far? I would like this possibility too, since compilation may consume alot of time and this way, we could greatly reduce deployment time.

@jeremy
Copy link

jeremy commented Apr 21, 2012

Nobody's working on it, @robertwachs. Please do investigate!

@steel
Copy link
Contributor

steel commented Apr 28, 2012

I have a fork that handles multi-threaded manifest.compile()
It's located at https://github.com/steel/sprockets/tree/multi-process-compiler

On an 8 core box running with 8 manifest compiler workers we've dropped asset compilation from 7 minutes (1 core intel box) down to 50 seconds. Not too shabby.

I haven't had time to cleanup the code and issue a pull request yet but it does work.

@josh josh closed this as completed May 16, 2012
@jjb
Copy link

jjb commented Jun 3, 2012

@steel awesome!

@ghost
Copy link

ghost commented Jun 4, 2012

Nice solution!
Is there any way using this with Rails 3.2.2 in a clean way? I tried to bundle it but sadly, ActionPack depends on sprockets ~> 2.1.2 so it wont use this version. Also tried to backport it but the structure of Sprockets changed alot.

If someone has a solution for this, please leave a note. Thanks

Edit Seems like @guilleiguaran took care of that: rails/rails#5409 but its not in 3.2 stable yet.

@steel
Copy link
Contributor

steel commented Jan 11, 2013

Boy its been a long time but finally dropped the monkey patch into a gem:

https://rubygems.org/gems/sprockets-derailleur

You can now compile your manifest with as many processes as you need and I've included a processor detector as well.

processes = SprocketsDerailleur::number_of_processors rescue 1
manifest = Sprockets::Manifest.new(Application::Sprockets, 'public/assets', processes)

We use this multiple times a day deploying all over the place. Hope this helps other out there.

@chamnap
Copy link

chamnap commented Nov 24, 2014

@steel awesome!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants