Skip to content
This repository
Newer
Older
100644 380 lines (277 sloc) 17.03 kb
1fe22b3a »
2013-06-11 Update README.md
1 # TAGG: Threads à gogo for Node.js
ab01f724 »
2011-11-22 2nd commit, D day
2
1a73ae39 »
2013-06-11 Update README.md
3 Threads à gogo ([*](http://en.wikipedia.org/wiki/%C3%80_gogo)) is a native module for Node.js that provides an asynchronous, evented and/or continuation passing style API for moving blocking/longish CPU-bound tasks out of Node's event loop to JavaScript threads that run in parallel in the background and that use all the available CPU cores automatically; all from within **a single Node** process.
ab01f724 »
2011-11-22 2nd commit, D day
4
5 ## Installing the module
6
7 With [npm](http://npmjs.org/):
8
9 npm install threads_a_gogo
10
11 From source:
12
13 git clone http://github.com/xk/node-threads-a-gogo.git
14 cd node-threads-a-gogo
b86afb55 » IngwiePhoenix
2014-02-27 Updating this nice library to be compatible with node-gyp.
15 node-gyp rebuild
16 # It also works with node-waf, but this is outdated, so please use node-gyp nowdays.
ab01f724 »
2011-11-22 2nd commit, D day
17
18 To include the module in your project:
19
20 var threads_a_gogo= require('threads_a_gogo');
21
22 **You need a node with a v8 >= 3.2.4 to run this module. Any node >= 0.5.1 comes with a v8 >= 3.2.4.**
23
24 The module **runs fine, though, in any node >= 0.2.0** as long as you build it with a v8 >= 3.2.4. To do that you simply have to replace /node/deps/v8 with a newer version of v8 and recompile it (node). To get any version of node goto http://nodejs.org/dist/, and for v8 goto http://github.com/v8/v8, click on "branch", select the proper tag (>= 3.2.4), and download the .zip.
25
1a73ae39 »
2013-06-11 Update README.md
26 ## Intro
ab01f724 »
2011-11-22 2nd commit, D day
27
28 After the initialization phase of a Node program, whose purpose is to setup listeners and callbacks to be executed in response to events, the next phase, the proper execution of the program, is orchestrated by the event loop whose duty is to [juggle events, listeners and callbacks quickly and without any hiccups nor interruptions that would ruin its performance](http://youtube.com/v/D0uA_NOb0PE?autoplay=1)
29
30 Both the event loop and said listeners and callbacks run sequentially in a single thread of execution, Node's main thread. If any of them ever blocks, nothing else will happen for the duration of the block: no more events will be handled, no more callbacks nor listeners nor timeouts nor nextTick()ed functions will have the chance to run and do their job, because they won't be called by the blocked event loop, and the program will turn sluggish at best, or appear to be frozen and dead at worst.
31
32 **A.-** Here's a program that makes Node's event loop spin freely and as fast as possible: it simply prints a dot to the console in each turn:
33
34 cat examples/quickIntro_loop.js
35
36 ``` javascript
37 (function spinForever () {
38 process.stdout.write(".");
39 process.nextTick(spinForever);
40 })();
41 ```
42
43 **B.-** Here's another program that adds to the one above a fibonacci(35) call in each turn, a CPU-bound task that takes quite a while to complete and that blocks the event loop making it spin slowly and clumsily. The point is simply to show that you can't put a job like that in the event loop because Node will stop performing properly when its event loop can't spin fast and freely due to a callback/listener/nextTick()ed function that's blocking.
44
45 cat examples/quickIntro_blocking.js
46
47 ``` javascript
48 function fibo (n) {
49 return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
50 }
51
52 (function fiboLoop () {
53 process.stdout.write(fibo(35).toString());
54 process.nextTick(fiboLoop);
55 })();
56
57 (function spinForever () {
58 process.stdout.write(".");
59 process.nextTick(spinForever);
60 })();
61 ```
62
63 **C.-** The program below uses `threads_a_gogo` to run the fibonacci(35) calls in a background thread, so Node's event loop isn't blocked at all and can spin freely again at full speed:
64
65 cat examples/quickIntro_oneThread.js
66
67 ``` javascript
68 function fibo (n) {
69 return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
70 }
71
72 function cb (err, data) {
73 process.stdout.write(data);
74 this.eval('fibo(35)', cb);
75 }
76
77 var thread= require('threads_a_gogo').create();
78
79 thread.eval(fibo).eval('fibo(35)', cb);
80
81 (function spinForever () {
82 process.stdout.write(".");
83 process.nextTick(spinForever);
84 })();
85 ```
86
87 **D.-** This example is almost identical to the one above, only that it creates 5 threads instead of one, each running a fibonacci(35) in parallel and in parallel too with Node's event loop that keeps spinning happily at full speed in its own thread:
88
89 cat examples/quickIntro_fiveThreads.js
90
91 ``` javascript
92 function fibo (n) {
93 return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
94 }
95
96 function cb (err, data) {
97 process.stdout.write(" ["+ this.id+ "]"+ data);
98 this.eval('fibo(35)', cb);
99 }
100
101 var threads_a_gogo= require('threads_a_gogo');
102
103 threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
104 threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
105 threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
106 threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
107 threads_a_gogo.create().eval(fibo).eval('fibo(35)', cb);
108
109 (function spinForever () {
110 process.stdout.write(".");
111 process.nextTick(spinForever);
112 })();
113 ```
114
115 **E.-** The next one asks `threads_a_gogo` to create a pool of 10 background threads, instead of creating them manually one by one:
116
117 cat examples/multiThread.js
118
119 ``` javascript
120 function fibo (n) {
121 return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
122 }
123
124 var numThreads= 10;
125 var threadPool= require('threads_a_gogo').createPool(numThreads).all.eval(fibo);
126
127 threadPool.all.eval('fibo(35)', function cb (err, data) {
128 process.stdout.write(" ["+ this.id+ "]"+ data);
129 this.eval('fibo(35)', cb);
130 });
131
132 (function spinForever () {
133 process.stdout.write(".");
134 process.nextTick(spinForever);
135 })();
136 ```
137
138 **F.-** This is a demo of the `threads_a_gogo` eventEmitter API, using one thread:
139
140 cat examples/quickIntro_oneThreadEvented.js
141
142 ``` javascript
143 var thread= require('threads_a_gogo').create();
144 thread.load(__dirname + '/quickIntro_evented_childThreadCode.js');
145
146 /*
147 This is the code that's .load()ed into the child/background thread:
148
149 function fibo (n) {
150 return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
151 }
152
153 thread.on('giveMeTheFibo', function onGiveMeTheFibo (data) {
154 this.emit('theFiboIs', fibo(+data)); //Emits 'theFiboIs' in the parent/main thread.
155 });
156
157 */
158
159 //Emit 'giveMeTheFibo' in the child/background thread.
160 thread.emit('giveMeTheFibo', 35);
161
162 //Listener for the 'theFiboIs' events emitted by the child/background thread.
163 thread.on('theFiboIs', function cb (data) {
164 process.stdout.write(data);
165 this.emit('giveMeTheFibo', 35);
166 });
167
168 (function spinForever () {
169 process.stdout.write(".");
170 process.nextTick(spinForever);
171 })();
172 ```
173
174 **G.-** This is a demo of the `threads_a_gogo` eventEmitter API, using a pool of threads:
175
176 cat examples/quickIntro_multiThreadEvented.js
177
178 ``` javascript
179 var numThreads= 10;
180 var threadPool= require('threads_a_gogo').createPool(numThreads);
181 threadPool.load(__dirname + '/quickIntro_evented_childThreadCode.js');
182
183 /*
184 This is the code that's .load()ed into the child/background threads:
185
186 function fibo (n) {
187 return n > 1 ? fibo(n - 1) + fibo(n - 2) : 1;
188 }
189
190 thread.on('giveMeTheFibo', function onGiveMeTheFibo (data) {
191 this.emit('theFiboIs', fibo(+data)); //Emits 'theFiboIs' in the parent/main thread.
192 });
193
194 */
195
196 //Emit 'giveMeTheFibo' in all the child/background threads.
197 threadPool.all.emit('giveMeTheFibo', 35);
198
199 //Listener for the 'theFiboIs' events emitted by the child/background threads.
200 threadPool.on('theFiboIs', function cb (data) {
201 process.stdout.write(" ["+ this.id+ "]"+ data);
202 this.emit('giveMeTheFibo', 35);
203 });
204
205 (function spinForever () {
206 process.stdout.write(".");
207 process.nextTick(spinForever);
208 })();
209 ```
210
211 ## More examples
212
213 The `examples` directory contains a few more examples:
214
215 * [ex01_basic](https://github.com/xk/node-threads-a-gogo/blob/master/examples/ex01_basic.md): Running a simple function in a thread.
216 * [ex02_events](https://github.com/xk/node-threads-a-gogo/blob/master/examples/ex02_events.md): Sending events from a worker thread.
217 * [ex03_ping_pong](https://github.com/xk/node-threads-a-gogo/blob/master/examples/ex03_ping_pong.md): Sending events both ways between the main thread and a worker thread.
218 * [ex04_main](https://github.com/xk/node-threads-a-gogo/blob/master/examples/ex04_main.md): Loading the worker code from a file.
219 * [ex05_pool](https://github.com/xk/node-threads-a-gogo/blob/master/examples/ex05_pool.md): Using the thread pool.
220 * [ex06_jason](https://github.com/xk/node-threads-a-gogo/blob/master/examples/ex06_jason.md): Passing complex objects to threads.
221
222 ## API
223
224 ### Module API
225 ``` javascript
226 var threads_a_gogo= require('threads_a_gogo');
227 ```
228 ##### .create()
229 `threads_a_gogo.create( /* no arguments */ )` -> thread object
230 ##### .createPool( numThreads )
231 `threads_a_gogo.createPool( numberOfThreads )` -> threadPool object
232
233 ***
234 ### Thread API
235 ``` javascript
236 var thread= threads_a_gogo.create();
237 ```
238 ##### .id
239 `thread.id` -> a sequential thread serial number
240 ##### .load( absolutePath [, cb] )
241 `thread.load( absolutePath [, cb] )` -> reads the file at `absolutePath` and `thread.eval(fileContents, cb)`.
242 ##### .eval( program [, cb])
243 `thread.eval( program [, cb])` -> converts `program.toString()` and eval()s it in the thread's global context, and (if provided) returns the completion value to `cb(err, completionValue)`.
244 ##### .on( eventType, listener )
245 `thread.on( eventType, listener )` -> registers the listener `listener(data)` for any events of `eventType` that the thread `thread` may emit.
246 ##### .once( eventType, listener )
247 `thread.once( eventType, listener )` -> like `thread.on()`, but the listener will only be called once.
248 ##### .removeAllListeners( [eventType] )
249 `thread.removeAllListeners( [eventType] )` -> deletes all listeners for all eventTypes. If `eventType` is provided, deletes all listeners only for the event type `eventType`.
250 ##### .emit( eventType, eventData [, eventData ... ] )
251 `thread.emit( eventType, eventData [, eventData ... ] )` -> emit an event of `eventType` with `eventData` inside the thread `thread`. All its arguments are .toString()ed.
252 ##### .destroy( /* no arguments */ )
253 `thread.destroy( /* no arguments */ )` -> destroys the thread.
254
255 ***
256 ### Thread pool API
257 ``` javascript
258 threadPool= threads_a_gogo.createPool( numberOfThreads );
259 ```
260 ##### .load( absolutePath [, cb] )
261 `threadPool.load( absolutePath [, cb] )` -> `thread.load( absolutePath [, cb] )` in all the pool's threads.
262 ##### .any.eval( program, cb )
263 `threadPool.any.eval( program, cb )` -> like `thread.eval()`, but in any of the pool's threads.
264 ##### .any.emit( eventType, eventData [, eventData ... ] )
265 `threadPool.any.emit( eventType, eventData [, eventData ... ] )` -> like `thread.emit()` but in any of the pool's threads.
266 ##### .all.eval( program, cb )
267 `threadPool.all.eval( program, cb )` -> like `thread.eval()`, but in all the pool's threads.
268 ##### .all.emit( eventType, eventData [, eventData ... ] )
269 `threadPool.all.emit( eventType, eventData [, eventData ... ] )` -> like `thread.emit()` but in all the pool's threads.
270 ##### .on( eventType, listener )
271 `threadPool.on( eventType, listener )` -> like `thread.on()`, registers listeners for events from any of the threads in the pool.
272 ##### .totalThreads()
273 `threadPool.totalThreads()` -> returns the number of threads in this pool: as supplied in `.createPool( number )`
274 ##### .idleThreads()
275 `threadPool.idleThreads()` -> returns the number of threads in this pool that are currently idle (sleeping)
276 ##### .pendingJobs()
277 `threadPool.pendingJobs()` -> returns the number of jobs pending.
278 ##### .destroy( [ rudely ] )
279 `threadPool.destroy( [ rudely ] )` -> waits until `pendingJobs()` is zero and then destroys the pool. If `rudely` is truthy, then it doesn't wait for `pendingJobs === 0`.
280
281 ***
282 ### Global thread API
283
284 Inside every thread .create()d by threads_a_gogo, there's a global `thread` object with these properties:
285 ##### .id
286 `thread.id` -> the serial number of this thread
287 ##### .on( eventType, listener )
288 `thread.on( eventType, listener )` -> just like `thread.on()` above.
289 ##### .once( eventType, listener )
290 `thread.once( eventType, listener )` -> just like `thread.once()` above.
291 ##### .emit( eventType, eventData [, eventData ... ] )
292 `thread.emit( eventType, eventData [, eventData ... ] )` -> just like `thread.emit()` above.
293 ##### .removeAllListeners( [eventType] )
294 `thread.removeAllListeners( [eventType] )` -> just like `thread.removeAllListeners()` above.
295 ##### .nextTick( function )
296 `thread.nextTick( function )` -> like `process.nextTick()`, but twice as fast.
297
298 ***
299 ### Global puts
300
301 Inside every thread .create()d by threads_a_gogo, there's a global `puts`:
302 ##### puts(arg1 [, arg2 ...])
303 `puts(arg1 [, arg2 ...])` -> .toString()s and prints its arguments to stdout.
304
305 ## Rationale
306
307 [Node.js](http://nodejs.org) is the most [awesome, cute and super-sexy](http://javascriptology.com/threads_a_gogo/sexy.jpg) piece of free, open source software.
308
309 Its event loop can spin as fast and smooth as a turbo, and roughly speaking, **the faster it spins, the more power it delivers**. That's why [@ryah](http://twitter.com/ryah) took great care to ensure that no -possibly slow- I/O operations could ever block it: a pool of background threads (thanks to [Marc Lehmann's libeio library](http://software.schmorp.de/pkg/libeio.html)) handle any blocking I/O calls in the background, in parallel.
310
311 In Node it's verboten to write a server like this:
312
313 ``` javascript
314 http.createServer(function (req,res) {
315 res.end( fs.readFileSync(path) );
316 }).listen(port);
317 ```
318 Because synchronous I/O calls **block the turbo**, and without proper boost, Node.js begins to stutter and behaves clumsily. To avoid it there's the asynchronous version of `.readFile()`, in continuation passing style, that takes a callback:
319
320 ``` javascript
321 fs.readfile(path, function cb (err, data) { /* ... */ });
322 ```
323
324 It's cool, we love it (*), and there's hundreds of ad hoc built-in functions like this in Node to help us deal with almost any variety of possibly slow, blocking I/O.
325
326 ### But what's with longish, CPU-bound tasks?
327
328 How do you avoid blocking the event loop, when the task at hand isn't I/O bound, and lasts more than a few fractions of a millisecond?
329
330 ``` javascript
331 http.createServer(function cb (req,res) {
332 res.end( fibonacci(40) );
333 }).listen(port);
334 ```
335
336 You simply can't, because there's no way... well, there wasn't before `threads_a_gogo`.
337
338 ### What is Threads A GoGo for Node.js
339
340 `threads_a_gogo` provides the asynchronous API for CPU-bound tasks that's missing in Node.js. Both in continuation passing style (callbacks), and in event emitter style (event listeners).
341
342 The same API Node uses to delegate a longish I/O task to a background (libeio) thread:
343
344 `asyncIOTask(what, cb);`
345
346 `threads_a_gogo` uses to delegate a longish CPU task to a background (JavaScript) thread:
347
348 `thread.eval(program, cb);`
349
350 So with `threads_a_gogo` you can write:
351
352 ``` javascript
353 http.createServer(function (req,res) {
354 thread.eval('fibonacci(40)', function cb (err, data) {
355 res.end(data);
356 });
357 }).listen(port);
358 ```
359
360 And it won't block the event loop because the `fibonacci(40)` will run in parallel in a separate background thread.
361
362
363 ### Why Threads
364
365 Threads (kernel threads) are very interesting creatures. They provide:
366
367 1.- Parallelism: All the threads run in parallel. On a single core processor, the CPU is switched rapidly back and forth among the threads providing the illusion that the threads are running in parallel, albeit on a slower CPU than the real one. With 10 compute-bound threads in a process, the threads would appear to be running in parallel, each one on a CPU with 1/10th the speed of the real CPU. On a multi-core processor, threads are truly running in parallel, and get time-sliced when the number of threads exceed the number of cores. So with 12 compute bound threads on a quad-core processor each thread will appear to run at 1/3rd of the nominal core speed.
368
369 2.- Fairness: No thread is more important than another, cores and CPU slices are fairly distributed among threads by the OS scheduler.
370
371 3.- Threads fully exploit all the available CPU resources in your system. On a loaded system running many tasks in many threads, the more cores there are, the faster the threads will complete. Automatically.
372
373 4.- The threads of a process share exactly the same address space, that of the process they belong to. Every thread can access every memory address within the process' address space. This is a very appropriate setup when the threads are actually part of the same job and are actively and closely cooperating with each other. Passing a reference to a chunk of data via a pointer is many orders of magnitude faster than transferring a copy of the data via IPC.
374
375
376 ### Why not multiple processes.
377
378 The "can't block the event loop" problem is inherent to Node's evented model. No matter how many Node processes you have running as a [Node-cluster](http://blog.nodejs.org/2011/10/04/an-easy-way-to-build-scalable-network-programs/), it won't solve its issues with CPU-bound tasks.
379
380 Launch a cluster of N Nodes running the example B (`quickIntro_blocking.js`) above, and all you'll get is N -instead of one- Nodes with their event loops blocked and showing a sluggish performance.
Something went wrong with that request. Please try again.