Libuv thread pool UV_THREADPOOL_SIZE in a node cluser setup

[xushuwei202]

How is the environment variable UV_THREADPOOL_SIZE used under Node Cluster setup? Suppose I have this variable set to 4 and there are 10 cluster worker processes, would there be 4 * 10 = 40 threads running?

[devsnek]

each node process will have at least UV_THREADPOOL_SIZE + v8_thread_pool_size + 1 threads running.

the current default for both UV_THREADPOOL_SIZE and v8_thread_pool_size is 4.

This answer only reflects the current version of node.js and may not be true for past or future versions.

[paambaati]

@devsnek What's a good number for UV_THREADPOOL_SIZE, and what does it depend on? When is it a good idea to tune this?

[davisjam]

@paambaati, here's a bit of an essay. Hope something in here is helpful.

Node.js uses the event loop to execute JavaScript and also performs some asynchronous operation orchestration there (e.g. sending/receiving network traffic, depending on OS support for true asynchronous operations as exposed via libuv).

The worker pool handles asynchronous I/O operations for which there is weak kernel support, namely file system operations (fs) and DNS operations (dns); and is also used for asynchronous CPU-bound work in Node.js core modules, namely compression (zlib) and cryptography (crypto).

Tuning UV_THREADPOOL_SIZE depends somewhat on your workload. If you do a lot of compression and encryption then you might want to aim closer to the number of CPUs on your deployment machine. If you do a lot of I/O then increasing the size may help throughput because you will do a better job of saturating the I/O layer. For an example of this, many file systems and databases will benefit from having more pending I/O requests since they can make more optimal decisions about which data to read next -- e.g. the classic elevator algorithm. The hardcoded upper bound on the size of the threadpool is 128, enforced down in libuv.

Note, however, that tuning UV_THREADPOOL_SIZE may make more sense for a standalone application like a CLI written in Node.js. If you are standing up a bunch of Node.js processes using the cluster module then I would be surprised if tuning UV_THREADPOOL_SIZE was particularly beneficial for you. But if your application resembles the web tooling benchmarks then tuning UV_THREADPOOL_SIZE may help with performance.

There is a pretty rich history in Node.js and libuv issues and PRs discussing whether and how to handle different classes of asynchronous operations most efficiently. The only concrete example to date was @addaleax's recent excellent contribution to reduce the incidence of DNS hogging the threadpool (libuv #1845). I've summarized a lot of the history in this issue and am working on this PR to enable exploring different policies in Node.js. Exciting!

If my "pluggable threadpool" PR gets traction in libuv then I anticipate some motion in this space soon to make Node.js's threadpool a lot smarter.

[paambaati]

@davisjam Thanks a lot for taking the time to write this; it is hard to get high-level information about UV_THREADPOOL_SIZE. If you don't mind, I have a few more questions —

If you do a lot of I/O ...

What kind of I/O is this? Disk? Network? All of the above? How do I go about identifying where my bottleneck (i.e. event loop blocks/waits) lies?

For example, I have an app that parses a ton of HTMLs using cheerio, and applies a bunch of selectors on them. My guess is that parsing/building the ASTs (especially for very large HTMLs), applying the selectors and JSON.parse are the 3 slowest items there. I'm wondering if tuning the UV_THREADPOOL_SIZE would give me benefits here, but I'd like to gather more data and go about this a little more scientifically.

[davisjam]

What kind of I/O is this? Disk? Network?

Within the Node.js framework, the I/O that goes to the threadpool is:

• Asynchronous FS accesses
• Asynchronous DNS queries (getnameinfo, getaddrinfo)

libuv handles network I/O (e.g. HTTP traffic) directly on the event loop using sockets and something like epoll (details depend on the OS).

Modules with asynchronous I/O-themed APIs (e.g. database modules) may define their own uses of the threadpool using C++ add-ons.

How do I go about identifying where my bottleneck...lies?

I think the most action on this topic is in the node-clinic project. However, without something in libuv like libuv #1815, I don't think you can get much information about the threadpool. I haven't tried their tools in a few months though.

My guess is that parsing/building the ASTs (especially for very large HTMLs), applying the selectors and JSON.parse are the 3 slowest items there. I'm wondering if tuning the UV_THREADPOOL_SIZE would give me benefits here,

JavaScript code is not executed on the libuv threadpool, so if this code is written in JavaScript then tuning the libuv threadpool won't help. You could use something like @addaleax's Worker Threads to offload work to helper threads (or use a higher-level module like workerpool. This might be more lightweight than using the cluster module, though I don't know if there are benchmarks to support that.

[paambaati]

We can close this. Thanks a lot for the detailed answers.

…

On Thu, Aug 30, 2018, 6:20 AM Jamie Davis ***@***.***> wrote: @paambaati <https://github.com/paambaati> Let me know if you have more questions. If not I guess we can close out this issue? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22468 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFIehPrBf1eqV4xT86RvK6key2DD8SM-ks5uVzbrgaJpZM4WIspC> .

[sophister]

each node process will have at least UV_THREADPOOL_SIZE + v8_thread_pool_size + 1 threads running.

the current default for both UV_THREADPOOL_SIZE and v8_thread_pool_size is 4.

This answer only reflects the current version of node.js and may not be true for past or future versions.

I've write a test js like this:

const fs = require('fs');


console.log(process.pid);
/*
fs.readFile('./a.txt', function(err, out){
        if(err){
                return console.error(err);
        }
        console.log(out);
});

*/
setInterval(function(){

//console.log('timer');

}, 1000);

and get the process.pid is 18335, and using pstree -p 18335 shows below:

[dev@izm5e8tnigcymjk3zksao6z ~]$ pstree -p 18335
node(18335)─┬─{node}(18336)
            ├─{node}(18337)
            ├─{node}(18338)
            ├─{node}(18339)
            ├─{node}(18340)
            └─{node}(18341)
[dev@izm5e8tnigcymjk3zksao6z ~]$

there are 6 threads. If V8 use 4 threads default, then where do the another threads come from?

[sam-github]

Threads are used for more than the threadpool. v8 does some of its work on threads, the inspector runs on a thread, etc.

[davisjam]

It's also worth noting that the libuv threadpool is only initialized when it is used for the first time. I don't know if any of the Node startup code uses it (?), so perhaps those threads would only appear if you used one of the corresponding Node APIs?

[sophister]

@davisjam when I uncomment the following code, 4 more threads will exist, and they seem to be the libuv threads.

const fs = require('fs');


console.log(process.pid);

fs.readFile('./a.txt', function(err, out){
        if(err){
                return console.error(err);
        }
        console.log(out);
});

setInterval(function(){

//console.log('timer');

}, 1000);

[davisjam]

@sophister OK, then it sounds like the libuv threadpool is not initialized until a user interacts with it. Here, the asynchronous call to fs.readFile goes into the threadpool and causes the libuv threads to be initialized.

[acilgit]

JavaScript code is not executed on the libuv threadpool, so if this code is written in JavaScript then tuning the libuv threadpool won't help. You could use something like @addaleax's Worker Threads to offload work to helper threads (or use a higher-level module like workerpool. This might be more lightweight than using the cluster module, though I don't know if there are benchmarks to support that.

that's really helpful, thanks a lot!