Add Concurrency.

[JeremyTCD]

• Multi-threading using workers.
• Multi-processing using a pool of INodeJSServices.

[DaniilSokolyuk]

@JeremyTCD
It seems to me that clients can implement multithreading themselves? There can be a huge number of different options, like:

• queue size
• worker max usages
• different strategies to reduce or increase the size of the pool

For multi-processing - cool, less overhead compared to a cluster

[JeremyTCD]

If OutOfProcessNodeJSServiceOptions.Parallelization is Parallelization.None (default), it'll work the same as right now. Clients can use that mode if they want to handle threading. I suspect not every user will be familiar enough with node to handle threading on their own though.

For multi-processing - cool, less overhead compared to a cluster

Yeah agreed, tried cluster it's really slow. Requests go to a master then get sent by IPC to workers, so 2x latency.

I think this implementation may be faster than basic worker threads. We'll see.

[codecov]

Codecov Report

Merging #52 into master will decrease coverage by 0.55%.
The diff coverage is 92.85%.

@@            Coverage Diff             @@
##           master      #52      +/-   ##
==========================================
- Coverage   97.36%   96.81%   -0.56%     
==========================================
  Files          17       18       +1     
  Lines         494      533      +39     
==========================================
+ Hits          481      516      +35     
- Misses         13       17       +4

Impacted Files	Coverage Δ
src/NodeJS/NodeJSServiceCollectionExtensions.cs	100% <100%> (ø)	⬆️
...s/OutOfProcess/OutOfProcessNodeJSServiceOptions.cs	100% <100%> (ø)	⬆️
...ntations/OutOfProcess/OutOfProcessNodeJSService.cs	97.93% <100%> (-0.03%)	⬇️
...tations/OutOfProcess/Http/HttpNodeJSPoolService.cs	85.18% <85.18%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a05059...daf3fcb. Read the comment docs.

[JeremyTCD]

Published 5.1.0 with out-of-the-box multi-process concurrency.

Decided not to implement multi-threading using workers:

• Tricky passing stuff to and from threads, without major changes to HttpServer.ts, can't pass everything via transferlist. Can't pass streams either.
• Not all Node.js modules are available within workers.
• Build tools aren't mature yet. Webpack's worker-loader can't inline worker threads.

API allows for adding it in future though.