Multi-core support

[balupton]

@Naatan has suggested we adopt a multi-core architecture:

#762 (comment)

For this to happen, we must first modularize all the things #445

---

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

[greduan]

I don't see a benefit in this? It would be faster but if we optimize I think we could achieve the same speed easier, no?

Multi-core I think would only hide the slowness. IMO.

[Naatan]

There is no way that optimizing would achieve the same speed as a multi-threaded approach. The simple fact remains that no matter how much you optimize docpad you're still waiting for a write to finish before you can start another.

[balupton]

The simple fact remains that no matter how much you optimize docpad you're still waiting for a write to finish before you can start another.

How true is this? We perform all of our file system operations asynchronously... which from my understanding gets around this, as asynchronous io = non-blocking io... right?

[Naatan]

Right, but it's still sending feedback over a single node thread. I don't know how async write is handled in node though, so I'd imagine you would have a better understanding on how this might affect things in the long run. I'd also imagine that there are areas besides writing files that could benefit from a multi-threaded approach.

Worth noting that if DocPad is already properly handling writes asynchronously then in theory it shouldn't be too hard to write up a test case to analyze the performance benefits of a multi-threaded approach for writes.

Once I've properly delved into DocPad's innards and have some time to spare I won't mind looking into some of the things I'm proposing myself.

[pflannery]

I don't believe that nodejs has quite yet implemented multi-core support. They did try but ran in to too many issues so they rolled back...Some work around's are mentioned here

I think one thing docpad could benefit from is using promises (currently available in harmony mode) because event queuing is implemented directly inside V8 (instead the current fallback implementation using processTick\setImmediate with nodejs).

I heard the es6 (harmony) gets released in December this year? though not sure that is a reality.

[balupton]

I'm doubtful this will provide any benefits.