workers: initial implementation

[petkaantonov]

Workers are light-weight processes that are backed by OS threads.

This is an initial implementation that is behind the flag --experimental-workers. It should have very little effect on other io.js components, even if the flag is enabled. The module can be required by doing var Worker = require('worker'); - see how the tests do it in test/workers folder.

---

What is currently implemented:

Worker constructor which takes an entry module and an optional options object. Currently (I have some ideas for more) the only option is keepAlive:

var worker = new Worker('worker.js', {keepAlive: false})

The option defaults to true, which means the worker thread will stay alive even when its event loop is completely empty. This is because you might not have
anything to send to the worker right away when a server is started for instance.

A worker object is an event emitter, with the events 'message', 'exit', and 'error' and public methods postMessage, terminate, ref and unref:

• 'message' event will receive currently a single argument, that is directly the message posted from the worker thread.
• 'exit' event is emitted when the worker thread exits and receives the exit code as argument.
• 'error' event is emitted when there is an uncaught exception in the worker which would abort a normal node process. The argument is the error thrown,
  if it's an error object, its builtin type, stack, name and message are retained in the copy. It will additionally contain extra properties that were in the
  original error, such as .code.
• postMessage([message]) where message is structured clonable* can be used to send a message to the worker thread. Additionally internal messaging support is implemented
  that doesn't need to copy data or go through JS. (* currently implemented as JSON to keep PR small, more details later)
• terminate([callback]) terminates the worker by using V8 TerminateExecution and uv_async_send, so the only thing that
  can defer a worker's termination is when it is executing C code which shouldn't contain any infinite loop or such :-).
• unref and ref work like timer's respective methods, however keep in mind that a worker will terminate if its owner terminates.

Inside a worker thread, several process-wide features are disabled for example setting the process' current working directory, title, environment variables or umask. Workers can still read these, but only the main thread can change them. Otherwise all core modules and javascript works normally inside a worker, including console.log.

Inside worker thread, the Worker constructor object is an event emitter, with the event 'message' and the method postMessage() to communicate with its owner (see tests for usage). The worker constructor can be used to construct nested workers with the worker being their owner. Passing messages from grand child to grand parent requires the child's parent to pass its message through, however transferable message ports can be implemented if use cases emerge.

Nested workers were implemented because Web Workers support them too (except Chrome) and I envision that a setup with ${CPU cores} amount of web servers with N amounts of sub-workers for each will be very useful.

2 new public process read-only properties are introduced process.isMainInstance and process.isWorkerInstance.

---

Advantages of workers over processes:

• Internal ITC (not passing JS objects) is ridiculously fast with the wait-free message channel
• User ITC is probably faster as well, not having the overhead of JSON parsing and serialiazing
  • And can also transfer more kinds of objects, such as Dates, Maps, Sets, objects with circular references...
  • Additionally ownership transfer of external resources like ArrayBuffers and external strings can be done without copying
  • Ownership transfer of uv handles will probably require support from uv so that handles can be detached from an event loop
    and adopted by another.
• Threads have less context-switch overhead, making it feasible to run many more workers than there are CPU cores
  which is more graceful in high load situations. And if we implement worker-cluster, there shouldn't be the need
  to use home-made round-robing scheduler.
• Free of problems like process.send not always being synchronous
• There is just one process to kill and no runaway processes

Disadvantages:

• Cannot rely on OS to free resources upon worker exit. However the architecture that
  enables freeing all resources upon Environment destruction is there and
  is already used to free all resources that a new worker that doesn't do
  anything but exit immediately allocates.
• C code maintainers need to think about thread-safety. However, there is generally
  little amount of shared resources as most stuff is tied to a v8 isolate or a uv event loop
  which are exclusive to a worker.

Compared to traditional threading:

• Workers are expensive to create, one does not simply spin up a worker to do 1 task and throw it away.
• Workers cannot share any JS memory (this is impossible in V8), to JS users they appear pretty much as isolated as real processes

---

To keep the PR size small (believe it or not, deep copying will take much more code than workers alone), objects are still copied by JSON serialization. I want to implement something like the structured clone with some differences as the spec has some seriously nasty parts that are not really needed in practice but force the implementation to be so slow as to defeat the whole point of using it in the first place.

So what cannot be implemented from the algorithm:

• The html5 objects like File (since we don't have those)
• Named properties of Arrays (inconsistent and extremely nasty to implement using v8's api)
• (implicit) If you have sparse array and define indexed properties in protototypes, then those values in prototypes are used in the resulting array. V8 has internal check that array and object prototypes are clean of indexed properties, but we don't have access to that.

What could be implemented but don't want to implement from the algorithm:

• Normal properties of Map and Set. This would be inconsistent and add more stuff to implement, but certainly doable.

What I mean by inconsistent is that copying map, set and array properties is inconsistent because the properties of Date, RegExp, Boolean, Number, and String objects are dropped.

---

To make reviewing this easier, a description of changes in each file follows:

• *test/workers/**_, _tools/test.py, Makefile
  • For testing. Run make test-workers. Worker tests are not ran as a part of make test
• worker.cc, worker.h, worker.js
  • Majority of the worker implementation is here
• util.h, util-inl.h, util.cc
  • Added STATIC_ASSERT
  • Added some threading helpers
  • Added a hack to get a char* from js string (TODO)
  • Added Remove method to ListHead
• smalloc.cc, smalloc.h
  • Extracted CallbackInfo class declaration to a header
  • Moved ALLOC_ID to env.h
• producer-consumer-queue.h
  • Modified folly's wait-free single-producer single-consumer queue.
• persistent-handle-cleanup.cc, persistent-handle-cleanup.h.
  • Because finalizers are not guaranteed to be called even when a v8 isolate is disposed, these implement PersistentHandleVisitor that is used to walk all unfinalized weak persistent handles to release their C resources upon Environment destruction
• notification-channel.cc, notification-channel.h
  • These work around uv_async_send unreliability when async handles need to be unreffed
• node-internals.h
  • Removed NodeInstanceData - the event loop code for worker became too different from the main thread event loop code for this to be useful
• node-contextify.cc
  • Extracted out ContextifyScript class declaration to a header. The diff looks horrible but there is no real changes
  • Added check to prevent cancellation of termination in EvalMachine method
• node.js
  • Added worker mode startup and cleanup registration for stdio and signal handles. (environment->CleanupHandles() is actually unused by main instance, but workers use it)
• node.cc
  • Added locking around process-wide methods. (If uv's rwlock still causes deadlocks in Windows 2003, it should be changed to mutex).
  • Added worker-only locking for module initialization, the ApiLock() is null for main instance but workers need it because module initialization isn't termination safe.
  • Made MakeCallback safe to call inside workers that are being abruptly terminated
  • Changed all exit calls to env->Exit() (which only exits the process if called from main thread)
  • Added an export for process._registerHandleCleanup()
  • Moved atexit from LoadEnvironment to Init (probably not the right place, but it cannot be in LoadEnvironment)
  • Added CreateEnvironment overload to be called from worker threads
  • Extracted Environment initialization to InitializeEnvironment
  • Removed all worker checks from StartNodeInstance and renamed it to RunMainInstance. Added a call to TerminateSubWorkers() before exit event is emitted.
  • Added ShutdownPlatform and delete platform at the end of Start method.
• handle_wrap.cc, handle_wrap.h
  • Added a way to register handle cleanups upon Environment destruction.
• env.h, env-inl.h
  • Introduced a ClassId enum which weak persistent wrappers can use so that they can be walked upon Environment destruction and cleaned up for sure.
  • Added some worker related methods
  • Added set_using_cares(), which means that in Environment destructor destroy_cares is called
  • Added CanCallIntoJs() method: main instance can always call into js but workers can be in the middle of termination in which case one cannot call into js anymore
  • Added Exit method which does process exit if called from main thread and worker termination if called from worker thread
• cares_wrap.cc
  • Added InitAresOnce to do one-time process-wide initialization
  • Added a call to env->set_using_cares() (see env.h above)
• async-wrap.cc
  • Made same changes to MakeCallback as were made to node::MakeCallback
• timers.js
  • Added handle cleanup registrations for timer handles so that when a worker Environment is destructed, the handles won't leak
• child_process.js
  • Added handle cleanup registration for the pipe handle so that when a worker Environment is destructed, the handle won't leak. The other handles probably leak too but this was the only one that is caught by current tests. There is currently an assertion in worker termination code for leaked handles.
• LICENSE
  • Added ProducerConsumerQueue's license

[mscdex]

What exactly is meant by "Named properties of Arrays?"

[mscdex]

Can this reuse the existing cached require('util')?

[petkaantonov]

Property that is a string or symbol as opposed to an integer (which are called indexed properties)

[domenic]

Epic feature :D.

Regarding structured clone, @dslomov-chromium had plans to bring that into the ES spec and presumably also into V8 directly. I am not sure what the status of that is though. It would be good to collaborate to avoid future back-compat breakage if we do ever get the chance to switch to V8's structured clone.

[bnoordhuis]

That's the ES7 typed objects strawman, isn't it?

[vkurchatkin]

https://github.com/dslomov-chromium/ecmascript-structured-clone

[petkaantonov]

@domenic Yea, the thing should have been implemented in V8 to begin with. Firefox for example implements it in SpiderMonkey, with callbacks for host objects.

Another solution can be patching V8 with some API additions that are already possible in v8 but just not done. E.g.:

• Array::HaveIndexedPropertiesInPrototypeChain
• Array::IsSparse()
• Object::GetOwnRealNamedProperties()
• Object::GetRealIndexedOwnProperty

Also what would be nice to have:

• Object::GetRealNamedOwnProperty

But of course most optimal solution is implementing structured clone in v8, the above additions would just enable avoiding completely ridiculous things like having to call GetPropertyNames() on an array.

[bnoordhuis]

Object::GetRealNamedOwnProperty

You should be able to emulate that like this:

Local<Object> object = /* ... */;
Local<String> key = /* ... */;
Local<Value> property = object->GetRealNamedProperty(key);
const bool is_own_property =
    !property.IsEmpty() && object->GetRealNamedPropertyInPrototypeChain(key).IsEmpty();

Array::HaveIndexedPropertiesInPrototypeChain

Local<Array> array = /* ... */;
const uint32_t index = /* ... */;
const bool is_from_prototype =
    !array->HasRealIndexedProperty(index) &&
    !array->HasIndexedLookupInterceptor() &&
    array->Has(index);

(Or just check HasRealIndexedProperty for each element in the prototype chain.)

[bnoordhuis]

Maybe this.once() so the closure doesn't stay around?

[petkaantonov]

@bnoordhuis The check should be much cheaper than emulation, I mean at the cost of calling HasRealIndexedProperty alone you could have already retrieved a real own indexed property. And only if it's undefined would you need to check if it's a hole, which is a rare case.

[bnoordhuis]

An O(n) operation per closed handle is kind of unfortunate...

[petkaantonov]

Yes I thought about that and we should consider changing this to a structure that allows faster removal

[petkaantonov]

Or if the HandleCleanup object is available to the registerer, removal is constant time

[alubbe]

Can we do anything to help out @petkaantonov? It would be awesome to play around with this!

[bnoordhuis]

@alubbe There are still some outstanding issues / comments. I don't think anyone will object if you adopt this PR and make the necessary changes. Just fork Petka's branch and add your changes on top.

[petkaantonov]

#2133 replaces this PR

[GnorTech]

For those who want to write Node.js code in multithread program: NW.js implemented this by enabling Node.js in Web Workers: https://nwjs.io/blog/v0.18.4/